July 8-10, 2024 | Patrick Bassner, Eduard Frankford, and Stephan Krusche
Iris is an AI-driven virtual tutor integrated into the interactive learning platform Artemis, designed to provide personalized, context-aware assistance to computer science students. It supports students in programming exercises by offering subtle hints or counter-questions rather than complete solutions, fostering independent problem-solving skills. Iris uses a Chain-of-Thought prompting strategy to generate responses, incorporating a role description for an "excellent tutor" and examples of meaningful answers through few-shot learning. It accesses the problem statement, student code, and automated feedback to provide tailored advice.
An empirical evaluation shows that students perceive Iris as effective because it understands their questions, provides relevant support, and contributes to the learning process. Students find Iris valuable for programming exercises and homework, and feel confident solving programming tasks in exams without it. Iris is seen as a complement to, rather than a replacement for, human tutors. It creates a safe space for students to ask questions without judgment.
The study addresses three research questions: how students perceive Iris' effectiveness, whether they feel more comfortable asking Iris questions than human tutors, and their subjective reliance on Iris. The survey results indicate a generally positive perception of Iris' ability to understand student queries and provide assistance. Students find interactions with Iris engaging and motivating, and feel comfortable asking questions without judgment. However, they do not view Iris as a complete replacement for human tutors.
The findings suggest that Iris is a valuable tool for programming exercises and learning, but students still feel confident in their ability to solve tasks in exams without it. The study highlights the importance of calibrated assistance in educational settings and the need for further research into optimizing Iris' context awareness and integration with student code. The study also acknowledges limitations, including potential biases in self-reported data and the need for further exploration of Iris' effectiveness in different educational contexts. Overall, Iris is seen as a helpful tool for practice and learning, but students value human interaction in specific contexts. Future research should explore the integration of different LLMs and strategies to enhance Iris' effectiveness and accessibility.Iris is an AI-driven virtual tutor integrated into the interactive learning platform Artemis, designed to provide personalized, context-aware assistance to computer science students. It supports students in programming exercises by offering subtle hints or counter-questions rather than complete solutions, fostering independent problem-solving skills. Iris uses a Chain-of-Thought prompting strategy to generate responses, incorporating a role description for an "excellent tutor" and examples of meaningful answers through few-shot learning. It accesses the problem statement, student code, and automated feedback to provide tailored advice.
An empirical evaluation shows that students perceive Iris as effective because it understands their questions, provides relevant support, and contributes to the learning process. Students find Iris valuable for programming exercises and homework, and feel confident solving programming tasks in exams without it. Iris is seen as a complement to, rather than a replacement for, human tutors. It creates a safe space for students to ask questions without judgment.
The study addresses three research questions: how students perceive Iris' effectiveness, whether they feel more comfortable asking Iris questions than human tutors, and their subjective reliance on Iris. The survey results indicate a generally positive perception of Iris' ability to understand student queries and provide assistance. Students find interactions with Iris engaging and motivating, and feel comfortable asking questions without judgment. However, they do not view Iris as a complete replacement for human tutors.
The findings suggest that Iris is a valuable tool for programming exercises and learning, but students still feel confident in their ability to solve tasks in exams without it. The study highlights the importance of calibrated assistance in educational settings and the need for further research into optimizing Iris' context awareness and integration with student code. The study also acknowledges limitations, including potential biases in self-reported data and the need for further exploration of Iris' effectiveness in different educational contexts. Overall, Iris is seen as a helpful tool for practice and learning, but students value human interaction in specific contexts. Future research should explore the integration of different LLMs and strategies to enhance Iris' effectiveness and accessibility.