Efficient Estimation of Word Representations in Vector Space

Efficient Estimation of Word Representations in Vector Space

7 Sep 2013 | Tomas Mikolov, Kai Chen, Greg Corrado, Jeffrey Dean
The paper introduces two novel model architectures for computing continuous vector representations of words from large datasets, aiming to improve the quality of word vectors while reducing computational costs. The authors propose the Continuous Bag-of-Words (CBOW) and Continuous Skip-gram models, which are simpler and more efficient than previous neural network models. These models are trained on a 1.6 billion-word dataset, achieving high accuracy in word similarity tasks. The paper also discusses the importance of multiple degrees of similarity in word representations and presents a comprehensive test set for measuring both syntactic and semantic regularities. The authors demonstrate that their models outperform existing techniques, particularly in tasks requiring subtle semantic relationships, such as machine translation and information retrieval. The paper concludes by highlighting the potential of these word vectors for future NLP applications and the availability of the code and additional word vectors for further research.The paper introduces two novel model architectures for computing continuous vector representations of words from large datasets, aiming to improve the quality of word vectors while reducing computational costs. The authors propose the Continuous Bag-of-Words (CBOW) and Continuous Skip-gram models, which are simpler and more efficient than previous neural network models. These models are trained on a 1.6 billion-word dataset, achieving high accuracy in word similarity tasks. The paper also discusses the importance of multiple degrees of similarity in word representations and presents a comprehensive test set for measuring both syntactic and semantic regularities. The authors demonstrate that their models outperform existing techniques, particularly in tasks requiring subtle semantic relationships, such as machine translation and information retrieval. The paper concludes by highlighting the potential of these word vectors for future NLP applications and the availability of the code and additional word vectors for further research.
Reach us at info@study.space