The paper provides an overview of the TREC Video Retrieval Evaluation (TREC Vid), an international benchmarking activity aimed at advancing research in video information retrieval. TREC Vid, which completed its fifth annual cycle in 2005, involves nearly 70 research organizations, universities, and consortia. The evaluation covers various tasks such as interactive and automatic manual searching for shots within a video corpus, automatic detection of semantic and low-level video features, shot boundary detection, and story boundary detection in broadcast TV news. The paper discusses the evolution of TREC Vid, its impact on research progress, and the challenges and benefits of benchmarking evaluation campaigns. It highlights the importance of system evaluation over user evaluation due to the high cost and complexity of user involvement. The paper also compares TREC Vid with other evaluation campaigns like FRGC and INEX, emphasizing the shared data, evaluation procedures, and metrics used in these campaigns. Finally, it concludes that while there are potential drawbacks, the overall impact of benchmarking evaluation campaigns like TREC Vid has been positive, fostering collaboration, innovation, and progress in the field of video information retrieval.The paper provides an overview of the TREC Video Retrieval Evaluation (TREC Vid), an international benchmarking activity aimed at advancing research in video information retrieval. TREC Vid, which completed its fifth annual cycle in 2005, involves nearly 70 research organizations, universities, and consortia. The evaluation covers various tasks such as interactive and automatic manual searching for shots within a video corpus, automatic detection of semantic and low-level video features, shot boundary detection, and story boundary detection in broadcast TV news. The paper discusses the evolution of TREC Vid, its impact on research progress, and the challenges and benefits of benchmarking evaluation campaigns. It highlights the importance of system evaluation over user evaluation due to the high cost and complexity of user involvement. The paper also compares TREC Vid with other evaluation campaigns like FRGC and INEX, emphasizing the shared data, evaluation procedures, and metrics used in these campaigns. Finally, it concludes that while there are potential drawbacks, the overall impact of benchmarking evaluation campaigns like TREC Vid has been positive, fostering collaboration, innovation, and progress in the field of video information retrieval.