May 11–16, 2024, Honolulu, HI, USA | Liudmila, Zavolokina, Kilian, Sprenkamp, Zoya, Katashinskaya, Daniel Gordon, Jones, Gerhard, Schwabe
The paper introduces ClarifAI, an automated tool designed to enhance critical thinking in news consumption by detecting and explaining propaganda. Inspired by Kahneman's dual-system theory, ClarifAI leverages Large Language Models (LLMs) to identify and analyze propaganda techniques in news articles. The tool aims to shift readers from intuitive, System 1 thinking to more analytical, System 2 thinking, thereby fostering deeper understanding and critical evaluation of news content.
The study's contributions are threefold: proposing the design of ClarifAI, demonstrating its effectiveness in encouraging critical reading through an online experiment, and emphasizing the value of explanations in promoting critical thinking. The research is grounded in dual system thinking and digital nudging, using the design science research methodology (DSRM) to develop a proof-of-concept prototype.
Key findings from the expert survey and online experiment include:
- **Propaganda Awareness**: Both the Light and Full versions of ClarifAI significantly increased participants' awareness of propaganda techniques.
- **Net Promoter Score (NPS)**: The Full version of ClarifAI received higher NPS scores, indicating greater user satisfaction and willingness to recommend the tool.
- **Thinking Mode**: Participants using the Full version showed a more balanced mix of intuitive and analytical thinking, suggesting a deeper engagement with the content.
- **Reading Time and Bias Perception**: The Full version also led to longer reading times and a more positive perception of news objectivity.
The study concludes that ClarifAI effectively enhances critical thinking in news consumption, providing practical tools and design insights for mitigating propaganda in digital news.The paper introduces ClarifAI, an automated tool designed to enhance critical thinking in news consumption by detecting and explaining propaganda. Inspired by Kahneman's dual-system theory, ClarifAI leverages Large Language Models (LLMs) to identify and analyze propaganda techniques in news articles. The tool aims to shift readers from intuitive, System 1 thinking to more analytical, System 2 thinking, thereby fostering deeper understanding and critical evaluation of news content.
The study's contributions are threefold: proposing the design of ClarifAI, demonstrating its effectiveness in encouraging critical reading through an online experiment, and emphasizing the value of explanations in promoting critical thinking. The research is grounded in dual system thinking and digital nudging, using the design science research methodology (DSRM) to develop a proof-of-concept prototype.
Key findings from the expert survey and online experiment include:
- **Propaganda Awareness**: Both the Light and Full versions of ClarifAI significantly increased participants' awareness of propaganda techniques.
- **Net Promoter Score (NPS)**: The Full version of ClarifAI received higher NPS scores, indicating greater user satisfaction and willingness to recommend the tool.
- **Thinking Mode**: Participants using the Full version showed a more balanced mix of intuitive and analytical thinking, suggesting a deeper engagement with the content.
- **Reading Time and Bias Perception**: The Full version also led to longer reading times and a more positive perception of news objectivity.
The study concludes that ClarifAI effectively enhances critical thinking in news consumption, providing practical tools and design insights for mitigating propaganda in digital news.