May 11-16, 2024 | Antonette Shibani, Simon Knight, Kirsty Kitto, Ajanie Karunanayake, Simon Buckingham Shum
Artificial Intelligence (AI) has become a common part of society, but a key challenge is ensuring that humans are equipped with the necessary critical thinking and AI literacy skills to interact with machines effectively. These skills are especially important for learners in the age of generative AI, where AI tools can demonstrate complex knowledge and abilities previously thought to be uniquely human. This paper explores the concept of critical learner interaction with AI, using both theoretical models and empirical data. Preliminary findings suggest a general lack of deep interaction with AI during the writing process. The study aims to develop a framework for understanding critical interaction with AI in the context of written assessments, focusing on how students use AI tools like ChatGPT to support their writing.
The study analyzed data from 49 student assignments in a graduate data science course. The data included students' self-reflections on their use of AI and their ChatGPT prompts and responses. The researchers developed a framework called Critical Interaction with AI for Writing (CIAW) to characterize students' critical interaction with AI during writing tasks. The framework includes five dimensions: Ideation and Planning, Information Seeking and Evaluation, Writing and Presentation, Personal Reflection on AI-assisted Learning, and Conversational Engagement. Each dimension is characterized by codes indicating the level of critical interaction (Deep, Shallow, or Absent).
The findings show that most students engaged in shallow or absent interaction with AI, with few demonstrating deep critical interaction. Students who did engage in critical interaction tended to do so in their self-reflections. The study highlights the importance of developing critical thinking skills when interacting with AI, as well as the need for curriculum and assessment design changes to incorporate more training and skill development for learners to work effectively with AI. The study also notes the limitations of current data collection methods and suggests future research to address these issues. Overall, the study emphasizes the importance of critical engagement with AI in the age of generative AI, as it enables learners to understand the capabilities and limitations of AI and to collaborate with it as a partner in cognition.Artificial Intelligence (AI) has become a common part of society, but a key challenge is ensuring that humans are equipped with the necessary critical thinking and AI literacy skills to interact with machines effectively. These skills are especially important for learners in the age of generative AI, where AI tools can demonstrate complex knowledge and abilities previously thought to be uniquely human. This paper explores the concept of critical learner interaction with AI, using both theoretical models and empirical data. Preliminary findings suggest a general lack of deep interaction with AI during the writing process. The study aims to develop a framework for understanding critical interaction with AI in the context of written assessments, focusing on how students use AI tools like ChatGPT to support their writing.
The study analyzed data from 49 student assignments in a graduate data science course. The data included students' self-reflections on their use of AI and their ChatGPT prompts and responses. The researchers developed a framework called Critical Interaction with AI for Writing (CIAW) to characterize students' critical interaction with AI during writing tasks. The framework includes five dimensions: Ideation and Planning, Information Seeking and Evaluation, Writing and Presentation, Personal Reflection on AI-assisted Learning, and Conversational Engagement. Each dimension is characterized by codes indicating the level of critical interaction (Deep, Shallow, or Absent).
The findings show that most students engaged in shallow or absent interaction with AI, with few demonstrating deep critical interaction. Students who did engage in critical interaction tended to do so in their self-reflections. The study highlights the importance of developing critical thinking skills when interacting with AI, as well as the need for curriculum and assessment design changes to incorporate more training and skill development for learners to work effectively with AI. The study also notes the limitations of current data collection methods and suggests future research to address these issues. Overall, the study emphasizes the importance of critical engagement with AI in the age of generative AI, as it enables learners to understand the capabilities and limitations of AI and to collaborate with it as a partner in cognition.