June 23-25 2014 | Marco Baroni and Georgiana Dinu and Germán Kruszewski
This paper compares count-based and context-predicting semantic vectors, showing that context-predicting models outperform count-based ones in various lexical semantics tasks. The study evaluates multiple parameter settings across a wide range of benchmarks, including semantic relatedness, synonym detection, concept categorization, selectional preferences, and analogy tasks. The results indicate that context-predicting models, which learn word vectors by maximizing the probability of their contexts, achieve superior performance compared to count-based models, which rely on co-occurrence counts. The predict models, such as those trained with the word2vec toolkit, are more effective in capturing semantic relationships and are robust to parameter variations. While count models have a long history and can perform well in some tasks, the predict models consistently outperform them, especially in tasks requiring capturing complex semantic relations. The study also highlights the importance of parameter tuning and the effectiveness of predict models in handling large datasets. The results suggest that context-predicting models are a more promising approach for distributional semantics, offering better performance and scalability. The paper concludes that predict models are a significant advancement in semantic representation, providing a more accurate and efficient way to capture word meanings and relationships.This paper compares count-based and context-predicting semantic vectors, showing that context-predicting models outperform count-based ones in various lexical semantics tasks. The study evaluates multiple parameter settings across a wide range of benchmarks, including semantic relatedness, synonym detection, concept categorization, selectional preferences, and analogy tasks. The results indicate that context-predicting models, which learn word vectors by maximizing the probability of their contexts, achieve superior performance compared to count-based models, which rely on co-occurrence counts. The predict models, such as those trained with the word2vec toolkit, are more effective in capturing semantic relationships and are robust to parameter variations. While count models have a long history and can perform well in some tasks, the predict models consistently outperform them, especially in tasks requiring capturing complex semantic relations. The study also highlights the importance of parameter tuning and the effectiveness of predict models in handling large datasets. The results suggest that context-predicting models are a more promising approach for distributional semantics, offering better performance and scalability. The paper concludes that predict models are a significant advancement in semantic representation, providing a more accurate and efficient way to capture word meanings and relationships.