2010 | Lykke, Marianne; Larsen, Birger; Lund, Haakon; Ingwersen, Peter
The poster discusses the development and characteristics of a new test collection designed to evaluate integrated search, which combines results from multiple sources into a single ranked list. The collection includes approximately 18,000 monographic records, 160,000 PDF papers and journal articles, and 275,000 abstracts with varied metadata and vocabularies from the physics domain. It features 65 tasks based on real work scenarios and corresponding graded relevance assessments. The collection aims to address the challenges of integrating different types of documents and metadata, providing a realistic and controlled environment for evaluating integrated search systems. The authors describe the collection's development process, document types, search tasks, and relevance assessments, highlighting its potential for both system-driven and user-oriented evaluations.The poster discusses the development and characteristics of a new test collection designed to evaluate integrated search, which combines results from multiple sources into a single ranked list. The collection includes approximately 18,000 monographic records, 160,000 PDF papers and journal articles, and 275,000 abstracts with varied metadata and vocabularies from the physics domain. It features 65 tasks based on real work scenarios and corresponding graded relevance assessments. The collection aims to address the challenges of integrating different types of documents and metadata, providing a realistic and controlled environment for evaluating integrated search systems. The authors describe the collection's development process, document types, search tasks, and relevance assessments, highlighting its potential for both system-driven and user-oriented evaluations.