An Autoregressive Text-to-Graph Framework for Joint Entity and Relation Extraction

An Autoregressive Text-to-Graph Framework for Joint Entity and Relation Extraction

2024 | Urchade Zaratiana, Nadi Tomeh, Pierre Holat, Thierry Charnois
This paper proposes an autoregressive text-to-graph framework for joint entity and relation extraction. The method frames the task as a conditional sequence generation problem, generating a linearized graph where nodes represent text spans and edges represent relation triplets. The model uses a transformer encoder-decoder architecture with a pointing mechanism on a dynamic vocabulary of spans and relation types, enabling the model to capture the structural characteristics and boundaries of entities and relations while grounding the generated output in the original text. Evaluation on benchmark datasets validates the effectiveness of the approach, demonstrating competitive results. The model achieves state-of-the-art results on CoNLL 2004 and SciERC, surpassing previous comparable models in terms of Entity F1 scores and Relation F1 scores. The model's architecture includes an encoder that processes the input text sequence and a decoder that generates the next token in the sequence, conditioned on the previously generated tokens and the input representation. The decoder uses self-attention and cross-attention to attend to relevant information from both the previously generated tokens and the input text. The model also incorporates constrained decoding to ensure the well-formedness of the output graph. The model is trained using sentence augmentation to reduce the risk of premature generation of the <EOS> token and improve recall. The model's performance is evaluated on three benchmark datasets: CoNLL 2004, SciERC, and ACE 05, demonstrating strong results across all datasets. The model's contributions include a novel method for joint entity and relation extraction, a transformer encoder-decoder architecture with a pointing mechanism, and extensive evaluations on benchmark datasets. The model's architecture and training process are detailed, along with ablation studies on the number of decoder layers, sentence augmentation, nucleus sampling, and sequence ordering. The model's performance is also analyzed through attention maps and structure embeddings, showing that the model effectively captures the structural characteristics of entities and relations. The model's results are compared with state-of-the-art methods, demonstrating its effectiveness in achieving competitive results.This paper proposes an autoregressive text-to-graph framework for joint entity and relation extraction. The method frames the task as a conditional sequence generation problem, generating a linearized graph where nodes represent text spans and edges represent relation triplets. The model uses a transformer encoder-decoder architecture with a pointing mechanism on a dynamic vocabulary of spans and relation types, enabling the model to capture the structural characteristics and boundaries of entities and relations while grounding the generated output in the original text. Evaluation on benchmark datasets validates the effectiveness of the approach, demonstrating competitive results. The model achieves state-of-the-art results on CoNLL 2004 and SciERC, surpassing previous comparable models in terms of Entity F1 scores and Relation F1 scores. The model's architecture includes an encoder that processes the input text sequence and a decoder that generates the next token in the sequence, conditioned on the previously generated tokens and the input representation. The decoder uses self-attention and cross-attention to attend to relevant information from both the previously generated tokens and the input text. The model also incorporates constrained decoding to ensure the well-formedness of the output graph. The model is trained using sentence augmentation to reduce the risk of premature generation of the <EOS> token and improve recall. The model's performance is evaluated on three benchmark datasets: CoNLL 2004, SciERC, and ACE 05, demonstrating strong results across all datasets. The model's contributions include a novel method for joint entity and relation extraction, a transformer encoder-decoder architecture with a pointing mechanism, and extensive evaluations on benchmark datasets. The model's architecture and training process are detailed, along with ablation studies on the number of decoder layers, sentence augmentation, nucleus sampling, and sequence ordering. The model's performance is also analyzed through attention maps and structure embeddings, showing that the model effectively captures the structural characteristics of entities and relations. The model's results are compared with state-of-the-art methods, demonstrating its effectiveness in achieving competitive results.
Reach us at info@study.space
[slides and audio] An Autoregressive Text-to-Graph Framework for Joint Entity and Relation Extraction