Task Supportive and Personalized Human-Large Language Model Interaction: A User Study

Task Supportive and Personalized Human-Large Language Model Interaction: A User Study

March 10-14, 2024 | Ben Wang, Jiqun Liu, Jamshed Karimnazarov, Nicolas Thompson
This study explores the design of supportive and personalized human-large language model (LLM) interactions, focusing on improving user experience and task completion when interacting with ChatGPT. The research investigates how integrating task context and user perceptions into human-LLM interactions can help users manage expectations, reduce cognitive load, refine prompts, and increase engagement. A user study was conducted with 16 participants, including college students and crowdsourced workers, who performed a variety of tasks ranging from creative writing to programming. The study developed a ChatGPT-like platform with three supportive functions: perception articulation, prompt suggestions, and conversation explanation. These functions were found to be effective in guiding users through the interaction process, helping them better understand and refine their prompts, and improving their overall experience with the system. The study also highlights the challenges faced by users in formulating prompts and interpreting search results, which are common issues in information seeking and interactive information retrieval. The findings suggest that the supportive functions significantly enhance user engagement and task completion, particularly for users with lower familiarity with the task or the system. However, the study also notes that some users, particularly crowd workers, may have misunderstood the study's purpose or faced challenges in formulating their own tasks. The research underscores the importance of designing proactive and user-centric systems that can adapt to diverse user needs and contexts. It also emphasizes the need for further research into evaluating human-LLM interactions and addressing the unique challenges faced by under-served users in the era of generative AI. The study contributes to the understanding of how to leverage LLMs for proactive information seeking systems and provides insights into the evaluation of human-LLM interactions from both system and user perspectives.This study explores the design of supportive and personalized human-large language model (LLM) interactions, focusing on improving user experience and task completion when interacting with ChatGPT. The research investigates how integrating task context and user perceptions into human-LLM interactions can help users manage expectations, reduce cognitive load, refine prompts, and increase engagement. A user study was conducted with 16 participants, including college students and crowdsourced workers, who performed a variety of tasks ranging from creative writing to programming. The study developed a ChatGPT-like platform with three supportive functions: perception articulation, prompt suggestions, and conversation explanation. These functions were found to be effective in guiding users through the interaction process, helping them better understand and refine their prompts, and improving their overall experience with the system. The study also highlights the challenges faced by users in formulating prompts and interpreting search results, which are common issues in information seeking and interactive information retrieval. The findings suggest that the supportive functions significantly enhance user engagement and task completion, particularly for users with lower familiarity with the task or the system. However, the study also notes that some users, particularly crowd workers, may have misunderstood the study's purpose or faced challenges in formulating their own tasks. The research underscores the importance of designing proactive and user-centric systems that can adapt to diverse user needs and contexts. It also emphasizes the need for further research into evaluating human-LLM interactions and addressing the unique challenges faced by under-served users in the era of generative AI. The study contributes to the understanding of how to leverage LLMs for proactive information seeking systems and provides insights into the evaluation of human-LLM interactions from both system and user perspectives.
Reach us at info@study.space