March 10-14, 2024, Sheffield, United Kingdom | Albatool Wazzan, Stephen MacNeil, Richard Souvenir
This paper compares traditional and LLM-based search engines for the task of image geolocation, focusing on user interactions and query formulation strategies. The study involved 60 participants who were randomly assigned to use either a traditional search engine or an LLM-based search engine. The results show that participants using traditional search engines outperformed those using LLM-based search engines in terms of accurate image geolocation. Participants using traditional search engines issued more queries and tended to add more terms to their initial queries, while those using LLM-based search engines issued longer, more natural language queries but had shorter search sessions and consistently rephrased their initial queries. Qualitative findings reveal that participants using LLM-based search engines struggled with effective query formulation, particularly in formulating queries in different languages and effectively communicating their intent to the LLM. The study highlights the need for further research on human-centered design of LLM interfaces to improve user interactions and query formulation strategies.This paper compares traditional and LLM-based search engines for the task of image geolocation, focusing on user interactions and query formulation strategies. The study involved 60 participants who were randomly assigned to use either a traditional search engine or an LLM-based search engine. The results show that participants using traditional search engines outperformed those using LLM-based search engines in terms of accurate image geolocation. Participants using traditional search engines issued more queries and tended to add more terms to their initial queries, while those using LLM-based search engines issued longer, more natural language queries but had shorter search sessions and consistently rephrased their initial queries. Qualitative findings reveal that participants using LLM-based search engines struggled with effective query formulation, particularly in formulating queries in different languages and effectively communicating their intent to the LLM. The study highlights the need for further research on human-centered design of LLM interfaces to improve user interactions and query formulation strategies.