Automated Learning of Decision Rules for Text Categorization

Automated Learning of Decision Rules for Text Categorization

July 1994 | CHIDANAND APTÉ and FRED DAMERAU, SHOLOM M. WEISS
This paper presents results from extensive experiments using optimized rule-based induction methods for text categorization. The goal is to automatically discover classification patterns for document categorization or personalized filtering of free text. The study compares machine-generated decision rules with human-engineered rule-based systems, showing that the former can match human performance using the same rule-based representation. On a key benchmark from the Reuters collection, results improved from a previously reported 67% recall/precision breakeven point to 80.5%. The study examines various methodological alternatives in high-dimensional feature spaces, including universal versus local dictionaries and binary versus frequency-related features. The paper describes an approach to automatically generating text categorization models. It discusses the process of inducing rule-based categorization models, including text representation, rule induction, and evaluation. The study shows that using local dictionaries for each classification topic can yield better results than universal dictionaries. It also demonstrates that using simple word counts as features can be effective, and that the rule induction method Swap-1 can generate accurate and efficient decision rules. The study evaluates the performance of different classification methods on the Reuters data set, finding that the best results were achieved with a local dictionary and frequency features, achieving a breakeven point of 80.5%. The study also compares the performance of machine-generated rules with human-engineered systems, finding that the former can achieve comparable results. The study concludes that optimized rule induction is competitive with other machine-learning techniques and can match human performance in text categorization. The results suggest that machine-learning methods can be effective for document classification, and that further research is needed to improve performance and handle more complex tasks.This paper presents results from extensive experiments using optimized rule-based induction methods for text categorization. The goal is to automatically discover classification patterns for document categorization or personalized filtering of free text. The study compares machine-generated decision rules with human-engineered rule-based systems, showing that the former can match human performance using the same rule-based representation. On a key benchmark from the Reuters collection, results improved from a previously reported 67% recall/precision breakeven point to 80.5%. The study examines various methodological alternatives in high-dimensional feature spaces, including universal versus local dictionaries and binary versus frequency-related features. The paper describes an approach to automatically generating text categorization models. It discusses the process of inducing rule-based categorization models, including text representation, rule induction, and evaluation. The study shows that using local dictionaries for each classification topic can yield better results than universal dictionaries. It also demonstrates that using simple word counts as features can be effective, and that the rule induction method Swap-1 can generate accurate and efficient decision rules. The study evaluates the performance of different classification methods on the Reuters data set, finding that the best results were achieved with a local dictionary and frequency features, achieving a breakeven point of 80.5%. The study also compares the performance of machine-generated rules with human-engineered systems, finding that the former can achieve comparable results. The study concludes that optimized rule induction is competitive with other machine-learning techniques and can match human performance in text categorization. The results suggest that machine-learning methods can be effective for document classification, and that further research is needed to improve performance and handle more complex tasks.
Reach us at info@study.space
[slides] Automated learning of decision rules for text categorization | StudySpace