SemEval-2024 Task 1: Semantic Textual Relatedness for African and Asian Languages

SemEval-2024 Task 1: Semantic Textual Relatedness for African and Asian Languages

2024 | Nedjma Ousidhoum, Shamsuddeen Hassan Muhammad, Mohamed Abdalla, Idris Abdulmumin, Ibrahim Said Ahmad, Sanchit Ahuja, Alham Fikri Aji, Vladimir Araujo, Meriem Beloucif, Christine De Kock, Oumaima Hourrane, Manish Shrivastava, Thamar Solorio, Nirmal Surange, Krishnapriya Vishnubhotla, Seid Muhie Yimam, Saif M. Mohammad
The paper presents the first shared task on Semantic Textual Relatedness (STR) for 14 African and Asian languages, focusing on the broader phenomenon of semantic relatedness rather than semantic similarity. The task includes three tracks: supervised, unsupervised, and cross-lingual. Participants were asked to rank sentence pairs by their semantic relatedness, with scores ranging from 0 (completely unrelated) to 1 (maximally related). The task attracted 163 participants, with 70 submissions from 51 teams. The evaluation metric was the Spearman rank correlation coefficient. The paper reports on the best-performing systems and common and effective approaches for each track. The results show that performance varies across languages and that pre-trained models do not always perform equally well. The paper also discusses the limitations and ethical considerations of the task.The paper presents the first shared task on Semantic Textual Relatedness (STR) for 14 African and Asian languages, focusing on the broader phenomenon of semantic relatedness rather than semantic similarity. The task includes three tracks: supervised, unsupervised, and cross-lingual. Participants were asked to rank sentence pairs by their semantic relatedness, with scores ranging from 0 (completely unrelated) to 1 (maximally related). The task attracted 163 participants, with 70 submissions from 51 teams. The evaluation metric was the Spearman rank correlation coefficient. The paper reports on the best-performing systems and common and effective approaches for each track. The results show that performance varies across languages and that pre-trained models do not always perform equally well. The paper also discusses the limitations and ethical considerations of the task.
Reach us at info@study.space
[slides] SemEval Task 1%3A Semantic Textual Relatedness for African and Asian Languages | StudySpace