SemEval 2024 – Task 10: Emotion Discovery and Reasoning its Flip in Conversation (EDIReF)

SemEval 2024 – Task 10: Emotion Discovery and Reasoning its Flip in Conversation (EDIReF)

29 Feb 2024 | Shivani Kumar¹, Md Shad Akhtar¹, Erik Cambria², Tanmoy Chakraborty³
SemEval 2024 Task 10: Emotion Discovery and Reasoning its Flip in Conversation (EDiReF) focuses on identifying emotions and understanding the reasons behind emotional shifts in monolingual English and Hindi-English code-mixed dialogues. The task includes three subtasks: emotion recognition in code-mixed dialogues, emotion flip reasoning for code-mixed dialogues, and emotion flip reasoning for English dialogues. A total of 84 participants engaged in the task, with the top systems achieving F1 scores of 0.70, 0.79, and 0.76 for the respective subtasks. The dataset includes manually annotated conversations focusing on emotions and triggers for emotional shifts. The task aims to assess the effectiveness of NLP systems in automatically addressing both emotion recognition and emotion flip reasoning. The datasets for the task include English and Hindi-English code-mixed conversations, with annotations for emotion labels and trigger utterances. The task was evaluated using F1 scores for emotion classification and trigger identification. The results show that LLMs and classical machine learning techniques were widely used, with some teams achieving high performance in emotion recognition and emotion flip reasoning. The findings highlight the challenges of code-mixing and the importance of context in emotion recognition. The task also emphasizes the need for further research in code-mixed language processing and the development of more effective models for emotion recognition and reasoning.SemEval 2024 Task 10: Emotion Discovery and Reasoning its Flip in Conversation (EDiReF) focuses on identifying emotions and understanding the reasons behind emotional shifts in monolingual English and Hindi-English code-mixed dialogues. The task includes three subtasks: emotion recognition in code-mixed dialogues, emotion flip reasoning for code-mixed dialogues, and emotion flip reasoning for English dialogues. A total of 84 participants engaged in the task, with the top systems achieving F1 scores of 0.70, 0.79, and 0.76 for the respective subtasks. The dataset includes manually annotated conversations focusing on emotions and triggers for emotional shifts. The task aims to assess the effectiveness of NLP systems in automatically addressing both emotion recognition and emotion flip reasoning. The datasets for the task include English and Hindi-English code-mixed conversations, with annotations for emotion labels and trigger utterances. The task was evaluated using F1 scores for emotion classification and trigger identification. The results show that LLMs and classical machine learning techniques were widely used, with some teams achieving high performance in emotion recognition and emotion flip reasoning. The findings highlight the challenges of code-mixing and the importance of context in emotion recognition. The task also emphasizes the need for further research in code-mixed language processing and the development of more effective models for emotion recognition and reasoning.
Reach us at info@study.space