A Markov Random Field Model for Term Dependencies

A Markov Random Field Model for Term Dependencies

August 15–19, 2005, Salvador, Brazil | Donald Metzler, W. Bruce Croft
This paper introduces a Markov Random Field (MRF) model for modeling term dependencies in information retrieval. The model allows for the incorporation of various text features, including single-term occurrences, ordered phrases, and unordered phrases. Three variants of the MRF model are explored: full independence (FI), sequential dependence (SD), and full dependence (FD). The FI variant assumes query terms are independent, while the SD variant models dependencies between adjacent query terms, and the FD variant captures dependencies between all subsets of query terms. The model is trained to maximize mean average precision rather than likelihood, and its effectiveness is evaluated on several newswire and web collections. Results show that the SD and FD variants significantly improve retrieval effectiveness, especially on larger web collections. The study also highlights the importance of considering multiple types of evidence and the impact of collection size on model performance.This paper introduces a Markov Random Field (MRF) model for modeling term dependencies in information retrieval. The model allows for the incorporation of various text features, including single-term occurrences, ordered phrases, and unordered phrases. Three variants of the MRF model are explored: full independence (FI), sequential dependence (SD), and full dependence (FD). The FI variant assumes query terms are independent, while the SD variant models dependencies between adjacent query terms, and the FD variant captures dependencies between all subsets of query terms. The model is trained to maximize mean average precision rather than likelihood, and its effectiveness is evaluated on several newswire and web collections. Results show that the SD and FD variants significantly improve retrieval effectiveness, especially on larger web collections. The study also highlights the importance of considering multiple types of evidence and the impact of collection size on model performance.
Reach us at info@study.space
[slides] A Markov random field model for term dependencies | StudySpace