SemEval-2014 Task 4: Aspect Based Sentiment Analysis

SemEval-2014 Task 4: Aspect Based Sentiment Analysis

August 23-24, 2014 | Maria Pontiki, Dimitrios Galanis, John Pavlopoulos, Haris Papageorgiou, Ion Androutsopoulos, Suresh Manandhar
SemEval-2014 Task 4 focused on aspect-based sentiment analysis (ABSA), aiming to identify aspects of target entities and the sentiment expressed for each. The task provided datasets of restaurant and laptop reviews, along with a common evaluation procedure, and attracted 163 submissions from 32 teams. The task included four subtasks: aspect term extraction (SB1), aspect term polarity (SB2), aspect category detection (SB3), and aspect category polarity (SB4). SB1 required identifying all aspect terms in reviews, including those with neutral polarity. SB2 determined the polarity of each aspect term. SB3 identified aspect categories (e.g., FOOD, PRICE) from reviews without annotations. SB4 determined the polarity of each aspect category. The datasets included 3041 restaurant sentences and 3845 laptop sentences. Annotations were done by human annotators using BRAT, a web-based tool. The task used F1 measure for SB1 and SB3, and accuracy for SB2 and SB4. Baselines included dictionary-based and similarity-based methods. Results showed that systems performed better in Phase B (SB3, SB4) than in Phase A (SB1, SB2). The best systems used CRF, SVM, and other machine learning techniques, leveraging training data and publicly available lexica. The NRC-Canada system achieved the highest scores in SB3 and SB4, while DCU and XRCE performed well in SB2. The task aimed to foster research in ABSA and provided benchmark datasets for two domains. Future work includes adding an aspect term aggregation subtask. The datasets are available through META-SHARE under a non-commercial license.SemEval-2014 Task 4 focused on aspect-based sentiment analysis (ABSA), aiming to identify aspects of target entities and the sentiment expressed for each. The task provided datasets of restaurant and laptop reviews, along with a common evaluation procedure, and attracted 163 submissions from 32 teams. The task included four subtasks: aspect term extraction (SB1), aspect term polarity (SB2), aspect category detection (SB3), and aspect category polarity (SB4). SB1 required identifying all aspect terms in reviews, including those with neutral polarity. SB2 determined the polarity of each aspect term. SB3 identified aspect categories (e.g., FOOD, PRICE) from reviews without annotations. SB4 determined the polarity of each aspect category. The datasets included 3041 restaurant sentences and 3845 laptop sentences. Annotations were done by human annotators using BRAT, a web-based tool. The task used F1 measure for SB1 and SB3, and accuracy for SB2 and SB4. Baselines included dictionary-based and similarity-based methods. Results showed that systems performed better in Phase B (SB3, SB4) than in Phase A (SB1, SB2). The best systems used CRF, SVM, and other machine learning techniques, leveraging training data and publicly available lexica. The NRC-Canada system achieved the highest scores in SB3 and SB4, while DCU and XRCE performed well in SB2. The task aimed to foster research in ABSA and provided benchmark datasets for two domains. Future work includes adding an aspect term aggregation subtask. The datasets are available through META-SHARE under a non-commercial license.
Reach us at info@study.space