July 2002 | Dmitry Zelenko, Chinatsu Aone, Anthony Richardella
This paper presents an application of kernel methods for extracting relations from unstructured natural language sources. The authors introduce kernels defined over shallow parse representations of text and design efficient algorithms for computing these kernels. They use these kernels in conjunction with Support Vector Machine (SVM) and Voted Perceptron learning algorithms to extract person-affiliation and organization-location relations from text. The proposed methods are experimentally evaluated and compared with feature-based learning algorithms, showing promising results.
The paper discusses the importance of shallow parsing in information extraction, as it provides a robust mechanism for producing text representations that can be effectively used for entity and relation extraction. The first step in their relation extraction approach is a powerful shallow parsing component of an information extraction system. The system comprises cascading finite state machines that identify names, noun phrases, and a restricted set of parts of speech in text. The system also classifies noun phrases and names as to whether they refer to people, organizations, and locations, thereby producing entities.
The authors formalize the relation extraction problem as a shallow parse classification problem. A shallow parse is turned into an example whose label reflects whether a relation of interest is expressed by the shallow parse. The learning system uses the labeled examples to output a model that is applied to shallow parses to obtain labels, and thus extract relations.
A unique property of the kernel methodology is that it does not explicitly generate features. Instead, examples retain their original representations and are used within learning algorithms only via computing a similarity (or kernel) function between them. This allows the learning system to implicitly explore a much larger feature space than one computationally feasible for processing with feature-based learning algorithms.
The paper also discusses related work on information extraction, including the use of probabilistic models such as Hidden Markov Models (HMM), Maximum Entropy Markov Models (MEMM), and Conditional Random Fields (CRF). It also discusses online learning algorithms for learning linear models, such as Perceptron and Winnow, which are becoming increasingly popular for NLP problems.
The authors introduce a class of kernel machine learning methods and apply them to relation extraction. They define kernels on parse trees and use them to compute similarity between shallow parse trees. They also define contiguous and sparse subtree kernels for relation extraction.
The paper presents an experimental evaluation of their approach, comparing it with feature-based linear methods. The results show that kernel methods exhibit excellent performance and fare better than feature-based algorithms in relation extraction. The results also highlight the importance of kernels, as algorithms with sparse subtree kernels are always significantly better than their contiguous counterparts.
The authors conclude that kernel-based machine learning methods are effective for extracting relations from natural language sources. They also plan to apply the kernel methodology to other sub-problems of information extraction, such as shallow parsing and entity extraction. They also plan to apply kernel methods for discourse processing, which involves collapsing entities, noun phrases, and pronouns into a set of equivalence classes.This paper presents an application of kernel methods for extracting relations from unstructured natural language sources. The authors introduce kernels defined over shallow parse representations of text and design efficient algorithms for computing these kernels. They use these kernels in conjunction with Support Vector Machine (SVM) and Voted Perceptron learning algorithms to extract person-affiliation and organization-location relations from text. The proposed methods are experimentally evaluated and compared with feature-based learning algorithms, showing promising results.
The paper discusses the importance of shallow parsing in information extraction, as it provides a robust mechanism for producing text representations that can be effectively used for entity and relation extraction. The first step in their relation extraction approach is a powerful shallow parsing component of an information extraction system. The system comprises cascading finite state machines that identify names, noun phrases, and a restricted set of parts of speech in text. The system also classifies noun phrases and names as to whether they refer to people, organizations, and locations, thereby producing entities.
The authors formalize the relation extraction problem as a shallow parse classification problem. A shallow parse is turned into an example whose label reflects whether a relation of interest is expressed by the shallow parse. The learning system uses the labeled examples to output a model that is applied to shallow parses to obtain labels, and thus extract relations.
A unique property of the kernel methodology is that it does not explicitly generate features. Instead, examples retain their original representations and are used within learning algorithms only via computing a similarity (or kernel) function between them. This allows the learning system to implicitly explore a much larger feature space than one computationally feasible for processing with feature-based learning algorithms.
The paper also discusses related work on information extraction, including the use of probabilistic models such as Hidden Markov Models (HMM), Maximum Entropy Markov Models (MEMM), and Conditional Random Fields (CRF). It also discusses online learning algorithms for learning linear models, such as Perceptron and Winnow, which are becoming increasingly popular for NLP problems.
The authors introduce a class of kernel machine learning methods and apply them to relation extraction. They define kernels on parse trees and use them to compute similarity between shallow parse trees. They also define contiguous and sparse subtree kernels for relation extraction.
The paper presents an experimental evaluation of their approach, comparing it with feature-based linear methods. The results show that kernel methods exhibit excellent performance and fare better than feature-based algorithms in relation extraction. The results also highlight the importance of kernels, as algorithms with sparse subtree kernels are always significantly better than their contiguous counterparts.
The authors conclude that kernel-based machine learning methods are effective for extracting relations from natural language sources. They also plan to apply the kernel methodology to other sub-problems of information extraction, such as shallow parsing and entity extraction. They also plan to apply kernel methods for discourse processing, which involves collapsing entities, noun phrases, and pronouns into a set of equivalence classes.