9 Jun 2015 | Oriol Vinyals, Lukasz Kaiser, Terry Koo, Slav Petrov, Ilya Sutskever, Geoffrey Hinton
This paper presents a novel approach to syntactic constituency parsing using a domain-agnostic attention-enhanced sequence-to-sequence model. The model achieves state-of-the-art results on the widely used Penn Treebank dataset when trained on a large synthetic corpus annotated with existing parsers. It also matches the performance of standard parsers when trained on a small human-annotated dataset, demonstrating high data efficiency. The parser is fast, processing over 100 sentences per second with an unoptimized CPU implementation. The paper discusses the architecture of the LSTM+A model, including the attention mechanism and linearization of parsing trees. Experiments show that the model generalizes well to different datasets and outperforms previous methods, highlighting the effectiveness of attention mechanisms in sequence-to-sequence tasks.This paper presents a novel approach to syntactic constituency parsing using a domain-agnostic attention-enhanced sequence-to-sequence model. The model achieves state-of-the-art results on the widely used Penn Treebank dataset when trained on a large synthetic corpus annotated with existing parsers. It also matches the performance of standard parsers when trained on a small human-annotated dataset, demonstrating high data efficiency. The parser is fast, processing over 100 sentences per second with an unoptimized CPU implementation. The paper discusses the architecture of the LSTM+A model, including the attention mechanism and linearization of parsing trees. Experiments show that the model generalizes well to different datasets and outperforms previous methods, highlighting the effectiveness of attention mechanisms in sequence-to-sequence tasks.