Is a Large Language Model a Good Annotator for Event Extraction?

Is a Large Language Model a Good Annotator for Event Extraction?

2024 | Ruirui Chen, Chengwei Qin, Weifeng Jiang, Dongkyu Choi
The paper explores the use of large language models (LLMs) as expert annotators for event extraction, a challenging task in natural language processing (NLP). Event extraction involves identifying and extracting specific events from text, which is often hampered by data scarcity and imbalance. The authors propose a novel approach where LLMs are used to generate additional labeled data, addressing these issues. By strategically including sample data from the training dataset in the prompt, they ensure that the distribution of LLM-generated samples aligns with the benchmark dataset. This method enhances the performance of fine-tuned models through data augmentation. Extensive experiments on popular benchmark datasets, such as ACE 2005 and MAVEN, demonstrate the effectiveness of this approach. The study also highlights the strengths and limitations of LLMs in event extraction, providing valuable insights for future research. The authors conclude that LLMs can significantly improve the performance of event extraction systems, particularly in real-world applications.The paper explores the use of large language models (LLMs) as expert annotators for event extraction, a challenging task in natural language processing (NLP). Event extraction involves identifying and extracting specific events from text, which is often hampered by data scarcity and imbalance. The authors propose a novel approach where LLMs are used to generate additional labeled data, addressing these issues. By strategically including sample data from the training dataset in the prompt, they ensure that the distribution of LLM-generated samples aligns with the benchmark dataset. This method enhances the performance of fine-tuned models through data augmentation. Extensive experiments on popular benchmark datasets, such as ACE 2005 and MAVEN, demonstrate the effectiveness of this approach. The study also highlights the strengths and limitations of LLMs in event extraction, providing valuable insights for future research. The authors conclude that LLMs can significantly improve the performance of event extraction systems, particularly in real-world applications.
Reach us at info@study.space
[slides and audio] Is a Large Language Model a Good Annotator for Event Extraction%3F