The thesis by Jeffrey Mitchell explores the composition of meanings within distributional models of semantics, focusing on how individual word meanings are combined to form representations of multi-word structures. Distributional models, which construct semantic representations from word usage patterns in large corpora, have been effective in cognitive modeling and practical applications such as semantic similarity judgments and association modeling. However, these models primarily focus on individual words rather than larger constructions like phrases and sentences. The thesis aims to address this gap by developing a framework for vector composition and evaluating it through three empirical tasks: modeling similarity judgments for short phrases, enhancing language models with semantic dependencies, and predicting reading times based on eye-movement data.
The framework includes existing methods like vector addition and tensor products, as well as novel composition functions such as simple multiplicative, weighted addition, and dilation models. These models are evaluated using natural data, showing that the simple multiplicative model performs well for word co-occurrence-based representations, while additive models are better for topic-based models. The thesis also demonstrates that compositional models improve performance over non-compositional models in language modeling and predicting processing difficulty in reading.
Overall, the research contributes to the understanding of semantic composition and provides practical tools for modeling complex linguistic structures.The thesis by Jeffrey Mitchell explores the composition of meanings within distributional models of semantics, focusing on how individual word meanings are combined to form representations of multi-word structures. Distributional models, which construct semantic representations from word usage patterns in large corpora, have been effective in cognitive modeling and practical applications such as semantic similarity judgments and association modeling. However, these models primarily focus on individual words rather than larger constructions like phrases and sentences. The thesis aims to address this gap by developing a framework for vector composition and evaluating it through three empirical tasks: modeling similarity judgments for short phrases, enhancing language models with semantic dependencies, and predicting reading times based on eye-movement data.
The framework includes existing methods like vector addition and tensor products, as well as novel composition functions such as simple multiplicative, weighted addition, and dilation models. These models are evaluated using natural data, showing that the simple multiplicative model performs well for word co-occurrence-based representations, while additive models are better for topic-based models. The thesis also demonstrates that compositional models improve performance over non-compositional models in language modeling and predicting processing difficulty in reading.
Overall, the research contributes to the understanding of semantic composition and provides practical tools for modeling complex linguistic structures.