FASTTEXT.ZIP: COMpressing TEXT CLASSIFICATION MODELS

FASTTEXT.ZIP: COMpressing TEXT CLASSIFICATION MODELS

12 Dec 2016 | Armand Joulin, Edouard Grave, Piotr Bojanowski, Matthijs Douze, Hervé Jégou & Tomas Mikolov
This paper presents a method for compressing text classification models to reduce memory usage while maintaining accuracy. The approach is based on product quantization (PQ) and other techniques to compress word embeddings and reduce the size of the model. The method is implemented as an extension of the fastText library, which is known for its efficiency in text classification. The proposed method achieves a significant reduction in memory usage, often by a factor of 1000, while only slightly sacrificing accuracy. The key components of the method include feature pruning, quantization, hashing, and retraining. The model is trained on several popular datasets and tested on multiple text classification benchmarks. The results show that the proposed method outperforms existing approaches in terms of the trade-off between memory usage and accuracy. The method is particularly effective for applications with limited memory, such as smartphones. The paper also discusses related work in the areas of text classification, language model compression, and neural network compression. The experiments demonstrate that the proposed method achieves high accuracy with significantly reduced memory usage, making it suitable for deployment in resource-constrained environments. The code and scripts for reproducing the results are made available as an extension of the fastText library.This paper presents a method for compressing text classification models to reduce memory usage while maintaining accuracy. The approach is based on product quantization (PQ) and other techniques to compress word embeddings and reduce the size of the model. The method is implemented as an extension of the fastText library, which is known for its efficiency in text classification. The proposed method achieves a significant reduction in memory usage, often by a factor of 1000, while only slightly sacrificing accuracy. The key components of the method include feature pruning, quantization, hashing, and retraining. The model is trained on several popular datasets and tested on multiple text classification benchmarks. The results show that the proposed method outperforms existing approaches in terms of the trade-off between memory usage and accuracy. The method is particularly effective for applications with limited memory, such as smartphones. The paper also discusses related work in the areas of text classification, language model compression, and neural network compression. The experiments demonstrate that the proposed method achieves high accuracy with significantly reduced memory usage, making it suitable for deployment in resource-constrained environments. The code and scripts for reproducing the results are made available as an extension of the fastText library.
Reach us at info@study.space
Understanding FastText.zip%3A Compressing text classification models