HILL: A Hallucination Identifier for Large Language Models Florian Leiser, Merlin Knaeble, Sven Eckhardt, Alexander Maedche, Valentin Leuthe, Gerhard Schwabe, and Ali Sunyaev present HILL, a user-centered artifact designed to help users identify hallucinations in large language models (LLMs). Hallucinations are defined as nonsensical, unfaithful, and undesirable text generated by LLMs. Users often rely too heavily on LLMs, leading to misinterpretations and errors. HILL was developed through a Wizard of Oz study with nine participants to identify key design features for an LLM interface. These features were then evaluated with 17 participants, and HILL's functionality was tested using a question-answering dataset and five user interviews. The results show that HILL can correctly identify and highlight hallucinations in LLM responses, enabling users to handle them with more caution. HILL is implemented as a web application that integrates with ChatGPT's API. It includes features such as a confidence score, source links, and a drill-down dashboard for detailed information. The artifact was evaluated through an online survey with 17 participants and performance validation using the Stanford Question Answering Dataset (SQuAD 2.0). The results indicate that HILL achieves an accuracy of 70.3% for answerable questions and 64.0% for unanswerable questions. The artifact also helps users assess the factual correctness of LLM responses, reducing overreliance on them. The study highlights the importance of user-centered design in AI artifacts and demonstrates how HILL can be adapted to existing LLMs to improve their reliability. The results show that HILL is effective in identifying hallucinations and providing users with the tools to assess the accuracy of LLM responses. The study also emphasizes the need for further research to improve the effectiveness of HILL and other similar artifacts in reducing overreliance on LLMs.HILL: A Hallucination Identifier for Large Language Models Florian Leiser, Merlin Knaeble, Sven Eckhardt, Alexander Maedche, Valentin Leuthe, Gerhard Schwabe, and Ali Sunyaev present HILL, a user-centered artifact designed to help users identify hallucinations in large language models (LLMs). Hallucinations are defined as nonsensical, unfaithful, and undesirable text generated by LLMs. Users often rely too heavily on LLMs, leading to misinterpretations and errors. HILL was developed through a Wizard of Oz study with nine participants to identify key design features for an LLM interface. These features were then evaluated with 17 participants, and HILL's functionality was tested using a question-answering dataset and five user interviews. The results show that HILL can correctly identify and highlight hallucinations in LLM responses, enabling users to handle them with more caution. HILL is implemented as a web application that integrates with ChatGPT's API. It includes features such as a confidence score, source links, and a drill-down dashboard for detailed information. The artifact was evaluated through an online survey with 17 participants and performance validation using the Stanford Question Answering Dataset (SQuAD 2.0). The results indicate that HILL achieves an accuracy of 70.3% for answerable questions and 64.0% for unanswerable questions. The artifact also helps users assess the factual correctness of LLM responses, reducing overreliance on them. The study highlights the importance of user-centered design in AI artifacts and demonstrates how HILL can be adapted to existing LLMs to improve their reliability. The results show that HILL is effective in identifying hallucinations and providing users with the tools to assess the accuracy of LLM responses. The study also emphasizes the need for further research to improve the effectiveness of HILL and other similar artifacts in reducing overreliance on LLMs.