The Automatic Content Extraction (ACE) Program Tasks, Data, and Evaluation

The Automatic Content Extraction (ACE) Program Tasks, Data, and Evaluation

1999 | George Doddington@NIST, Alexis Mitchell@LDC, Mark Przybocki@NIST, Lance Ramshaw@BBN, Stephanie Strassel@LDC, Ralph Weischedel@BBN
The Automatic Content Extraction (ACE) program aims to develop technology to automatically infer entities, relations, and events from human language data. The program involves defining research tasks, collecting and annotating data, and supporting research with evaluation tools and workshops. It began in 1999 with a pilot study and is scheduled for evaluation in September 2004. The ACE program focuses on three main tasks: entity detection and tracking, relation detection and characterization, and event detection and characterization. These tasks involve identifying entities, relations between entities, and events in which entities participate. The program includes data sources such as text, audio, and image data in multiple languages. The ACE program is a "technocentric" effort, emphasizing the development of core enabling technologies. The program's data is represented in XML format, with entities, relations, and events defined by their attributes and constituents. Annotation tasks include entity detection and tracking, relation detection and characterization, and event detection and characterization. The annotation process involves multiple passes over data, with a second pass conducted by more experienced annotators. The program also includes a cross-document evaluation of entities, relations, and events. The evaluation scoring system measures the accuracy of system outputs in detecting and recognizing entities, relations, and events. The scoring is based on the correct detection of target objects and their attributes, with penalties for false detections and incorrect attribute determination. The ACE program is supported by the Linguistic Data Consortium (LDC) and the National Institute of Standards and Technology (NIST), with data sources including broadcast news, newspaper, and newswire data. The program's evaluation includes a detailed scoring system for entities, relations, and events, with scores based on the correct identification of target objects and their attributes.The Automatic Content Extraction (ACE) program aims to develop technology to automatically infer entities, relations, and events from human language data. The program involves defining research tasks, collecting and annotating data, and supporting research with evaluation tools and workshops. It began in 1999 with a pilot study and is scheduled for evaluation in September 2004. The ACE program focuses on three main tasks: entity detection and tracking, relation detection and characterization, and event detection and characterization. These tasks involve identifying entities, relations between entities, and events in which entities participate. The program includes data sources such as text, audio, and image data in multiple languages. The ACE program is a "technocentric" effort, emphasizing the development of core enabling technologies. The program's data is represented in XML format, with entities, relations, and events defined by their attributes and constituents. Annotation tasks include entity detection and tracking, relation detection and characterization, and event detection and characterization. The annotation process involves multiple passes over data, with a second pass conducted by more experienced annotators. The program also includes a cross-document evaluation of entities, relations, and events. The evaluation scoring system measures the accuracy of system outputs in detecting and recognizing entities, relations, and events. The scoring is based on the correct detection of target objects and their attributes, with penalties for false detections and incorrect attribute determination. The ACE program is supported by the Linguistic Data Consortium (LDC) and the National Institute of Standards and Technology (NIST), with data sources including broadcast news, newspaper, and newswire data. The program's evaluation includes a detailed scoring system for entities, relations, and events, with scores based on the correct identification of target objects and their attributes.
Reach us at info@study.space