Desirable Characteristics for AI Teaching Assistants in Programming Education

Desirable Characteristics for AI Teaching Assistants in Programming Education

July 8-10, 2024 | Paul Denny, Stephen MacNeil, Jaromir Savelka, Leo Porter, Andrew Luxton-Reilly
This study explores the desirable characteristics of AI teaching assistants in programming education, focusing on how students perceive and value digital teaching assistants (TAs) in an introductory programming course. The research involved deploying an LLM-powered digital assistant called CodeHelp, which was used to provide support to 813 students. Students were asked to evaluate the quality of feedback and reflect on the characteristics they found most important in a digital TA. The study highlights that students value digital TAs for their ability to provide instant, engaging support, especially during peak times such as before assessment deadlines. They also expressed a strong preference for features that enable them to retain autonomy in their learning journey, such as scaffolding that helps guide them through problem-solving steps rather than simply being shown direct solutions. Key themes identified in student feedback included scaffolding, appropriateness, and user experience. Students emphasized the importance of explanations that focus on the process and foundational knowledge, rather than direct solutions. They also valued explanations that are appropriate for their level of expertise and presented in plain language. Additionally, students desired a user experience that allows them to take agency in their learning, with responses that are correct, clear, and concise. The study found that students generally found CodeHelp's responses to be correct and helpful, with a large majority agreeing that the responses were both correct and helpful. The research also highlights the importance of ensuring that digital TAs support healthy learning interactions and do not lead to over-reliance on the tools. The findings suggest that digital TAs can be a valuable complement to human TAs, especially in providing support outside of regular working hours. However, it is crucial for instructors to continue emphasizing the purpose and value of learning activities and assignments to ensure ethical use of AI TAs in the learning process.This study explores the desirable characteristics of AI teaching assistants in programming education, focusing on how students perceive and value digital teaching assistants (TAs) in an introductory programming course. The research involved deploying an LLM-powered digital assistant called CodeHelp, which was used to provide support to 813 students. Students were asked to evaluate the quality of feedback and reflect on the characteristics they found most important in a digital TA. The study highlights that students value digital TAs for their ability to provide instant, engaging support, especially during peak times such as before assessment deadlines. They also expressed a strong preference for features that enable them to retain autonomy in their learning journey, such as scaffolding that helps guide them through problem-solving steps rather than simply being shown direct solutions. Key themes identified in student feedback included scaffolding, appropriateness, and user experience. Students emphasized the importance of explanations that focus on the process and foundational knowledge, rather than direct solutions. They also valued explanations that are appropriate for their level of expertise and presented in plain language. Additionally, students desired a user experience that allows them to take agency in their learning, with responses that are correct, clear, and concise. The study found that students generally found CodeHelp's responses to be correct and helpful, with a large majority agreeing that the responses were both correct and helpful. The research also highlights the importance of ensuring that digital TAs support healthy learning interactions and do not lead to over-reliance on the tools. The findings suggest that digital TAs can be a valuable complement to human TAs, especially in providing support outside of regular working hours. However, it is crucial for instructors to continue emphasizing the purpose and value of learning activities and assignments to ensure ethical use of AI TAs in the learning process.
Reach us at info@study.space
Understanding Desirable Characteristics for AI Teaching Assistants in Programming Education