14 Oct 2024 | Yida Chen, Aoyu Wu, Trevor DePodesta, Catherine Yeh, Kenneth Li, Nicholas Castillo Marin, Oam Patel, Jan Riecke, Shivam Raval, Olivia Seow, Martin Wattenberg, Fernanda Viégas
The paper "Designing a Dashboard for Transparency and Control of Conversational AI" addresses the issue of transparency in conversational AI systems, particularly Large Language Models (LLMs). The authors present an end-to-end prototype that combines interpretability techniques with user experience design to make chatbots more transparent. They identify a "user model" within the LLM, which includes attributes such as age, gender, education level, and socioeconomic status. The prototype includes a dashboard that displays this user model in real time and allows users to control it. A user study is conducted to evaluate the prototype, and the results suggest that users appreciate the dashboard, which helps them understand biased behavior and increases their sense of control. The study also highlights user concerns about privacy and the need for further improvements in the dashboard's design. The authors conclude that their prototype provides evidence of a design pathway toward more transparent AI systems and emphasizes the importance of user research in interpretability.The paper "Designing a Dashboard for Transparency and Control of Conversational AI" addresses the issue of transparency in conversational AI systems, particularly Large Language Models (LLMs). The authors present an end-to-end prototype that combines interpretability techniques with user experience design to make chatbots more transparent. They identify a "user model" within the LLM, which includes attributes such as age, gender, education level, and socioeconomic status. The prototype includes a dashboard that displays this user model in real time and allows users to control it. A user study is conducted to evaluate the prototype, and the results suggest that users appreciate the dashboard, which helps them understand biased behavior and increases their sense of control. The study also highlights user concerns about privacy and the need for further improvements in the dashboard's design. The authors conclude that their prototype provides evidence of a design pathway toward more transparent AI systems and emphasizes the importance of user research in interpretability.