A Fast and Accurate Dependency Parser using Neural Networks

A Fast and Accurate Dependency Parser using Neural Networks

October 25-29, 2014 | Danqi Chen, Christopher D. Manning
This paper presents a fast and accurate dependency parser using neural networks. The authors propose a novel approach to learning a neural network classifier for use in a greedy, transition-based dependency parser. Instead of using sparse indicator features, which are common in current parsers, the method uses dense features, which are more efficient and effective. The neural network learns compact dense vector representations of words, part-of-speech (POS) tags, and dependency labels. This results in a fast, compact classifier that achieves a 2% improvement in unlabeled and labeled attachment scores on both English and Chinese datasets. The parser is based on the arc-standard system, which uses a stack, a buffer, and a set of dependency arcs to derive a target dependency parse tree. The parser uses a neural network to predict the correct transition based on features extracted from the configuration. The neural network uses a cube activation function, which allows it to model higher-order interaction features more effectively than traditional activation functions like tanh or sigmoid. The authors also introduce a novel activation function, the cube function, which is used to model the product terms of three different elements at the input layer. This function is more effective at capturing the interaction of three elements, which is a key property of dependency parsing. The parser is trained using a set of dense features, which are derived from the stack, buffer, and dependency arcs. The features include single-word features, word-pair features, and three-word features. The parser is able to parse over 1000 sentences per second at 92.2% unlabeled attachment score on the English Penn Treebank. The authors also compare their parser with other parsers, including Malt-Parser and MSTPArser. The results show that their parser achieves a 2% improvement in accuracy and is significantly faster than these parsers. The parser is also able to achieve better accuracy than Malt-Parser, which is known to be highly optimized. The authors also analyze the effects of different parser components, including the cube activation function, pre-trained word embeddings, and POS tag and arc label embeddings. The results show that these components contribute to the performance of the parser. The paper concludes that the proposed neural network parser is a fast and accurate method for dependency parsing, and that it outperforms other parsers in both accuracy and speed. The model is able to automatically learn the most useful feature conjunctions for making predictions, instead of hand-crafting them as indicator features.This paper presents a fast and accurate dependency parser using neural networks. The authors propose a novel approach to learning a neural network classifier for use in a greedy, transition-based dependency parser. Instead of using sparse indicator features, which are common in current parsers, the method uses dense features, which are more efficient and effective. The neural network learns compact dense vector representations of words, part-of-speech (POS) tags, and dependency labels. This results in a fast, compact classifier that achieves a 2% improvement in unlabeled and labeled attachment scores on both English and Chinese datasets. The parser is based on the arc-standard system, which uses a stack, a buffer, and a set of dependency arcs to derive a target dependency parse tree. The parser uses a neural network to predict the correct transition based on features extracted from the configuration. The neural network uses a cube activation function, which allows it to model higher-order interaction features more effectively than traditional activation functions like tanh or sigmoid. The authors also introduce a novel activation function, the cube function, which is used to model the product terms of three different elements at the input layer. This function is more effective at capturing the interaction of three elements, which is a key property of dependency parsing. The parser is trained using a set of dense features, which are derived from the stack, buffer, and dependency arcs. The features include single-word features, word-pair features, and three-word features. The parser is able to parse over 1000 sentences per second at 92.2% unlabeled attachment score on the English Penn Treebank. The authors also compare their parser with other parsers, including Malt-Parser and MSTPArser. The results show that their parser achieves a 2% improvement in accuracy and is significantly faster than these parsers. The parser is also able to achieve better accuracy than Malt-Parser, which is known to be highly optimized. The authors also analyze the effects of different parser components, including the cube activation function, pre-trained word embeddings, and POS tag and arc label embeddings. The results show that these components contribute to the performance of the parser. The paper concludes that the proposed neural network parser is a fast and accurate method for dependency parsing, and that it outperforms other parsers in both accuracy and speed. The model is able to automatically learn the most useful feature conjunctions for making predictions, instead of hand-crafting them as indicator features.
Reach us at info@study.space