4 Jan 2021 | Shervin Minaee, Nal Kalchbrenner, Erik Cambria, Narjes Nikzad, Meysam Chenaghlou, Jianfeng Gao
This paper provides a comprehensive review of over 150 deep learning (DL) models for text classification, including sentiment analysis, news categorization, topic classification, question answering (QA), and natural language inference (NLI). The authors categorize these models based on their neural network architectures, such as recurrent neural networks (RNNs), convolutional neural networks (CNNs), attention mechanisms, Transformers, and Capsule Nets. They also review over 40 popular text classification datasets and provide a quantitative analysis of the performance of selected DL models on 16 benchmarks. The paper discusses the main challenges and future directions for DL-based text classification.
Text classification involves assigning labels or tags to textual units such as sentences, queries, paragraphs, and documents. It has a wide range of applications including question answering, spam detection, sentiment analysis, news categorization, user intent classification, and content moderation. Text data can come from various sources, including web data, emails, chats, social media, tickets, insurance claims, user reviews, and questions and answers from customer services.
The paper discusses the evolution of text classification methods, starting with rule-based methods and machine learning (data-driven) based methods. It highlights the limitations of traditional machine learning approaches, such as reliance on hand-crafted features and the need for extensive feature engineering. Neural approaches, such as embedding models, have been explored to address these limitations. The paper reviews various embedding models, including latent semantic analysis (LSA), word2vec, ELMo, GPT, and BERT, and discusses their performance on various NLP tasks.
The paper also discusses different types of DL models for text classification, including feed-forward networks, RNN-based models, CNN-based models, attention mechanisms, memory-augmented networks, graph neural networks, and hybrid models. It highlights the strengths and weaknesses of each model and discusses their performance on various benchmarks. The paper concludes with a discussion of the main challenges and future directions for DL-based text classification.This paper provides a comprehensive review of over 150 deep learning (DL) models for text classification, including sentiment analysis, news categorization, topic classification, question answering (QA), and natural language inference (NLI). The authors categorize these models based on their neural network architectures, such as recurrent neural networks (RNNs), convolutional neural networks (CNNs), attention mechanisms, Transformers, and Capsule Nets. They also review over 40 popular text classification datasets and provide a quantitative analysis of the performance of selected DL models on 16 benchmarks. The paper discusses the main challenges and future directions for DL-based text classification.
Text classification involves assigning labels or tags to textual units such as sentences, queries, paragraphs, and documents. It has a wide range of applications including question answering, spam detection, sentiment analysis, news categorization, user intent classification, and content moderation. Text data can come from various sources, including web data, emails, chats, social media, tickets, insurance claims, user reviews, and questions and answers from customer services.
The paper discusses the evolution of text classification methods, starting with rule-based methods and machine learning (data-driven) based methods. It highlights the limitations of traditional machine learning approaches, such as reliance on hand-crafted features and the need for extensive feature engineering. Neural approaches, such as embedding models, have been explored to address these limitations. The paper reviews various embedding models, including latent semantic analysis (LSA), word2vec, ELMo, GPT, and BERT, and discusses their performance on various NLP tasks.
The paper also discusses different types of DL models for text classification, including feed-forward networks, RNN-based models, CNN-based models, attention mechanisms, memory-augmented networks, graph neural networks, and hybrid models. It highlights the strengths and weaknesses of each model and discusses their performance on various benchmarks. The paper concludes with a discussion of the main challenges and future directions for DL-based text classification.