11 Jun 2024 | Evanfiya Logacheva, Arto Hellas, James Prather, Sami Sarsa, Juho Leinonen
This article evaluates the quality and effectiveness of contextually personalized programming exercises generated by GPT-4 in an introductory programming course. The study aims to address the time-consuming and labor-intensive task of creating personalized exercises for students, which is a common challenge for computer science educators. The research is grounded in educational psychology, which suggests that context personalization can enhance students' situational interest and engagement. The study involves both expert and student evaluations of the generated exercises, as well as an analysis of student interactions with the system.
Key findings include:
- **Quality of Exercises**: Both experts and students rated the exercises highly, indicating that GPT-4-generated exercises are generally of good quality.
- **Student Engagement**: Students found the exercises engaging and useful, suggesting that AI-generated exercises can effectively supplement traditional teaching methods.
- **Contextual Personalization**: Students preferred choosing the theme of the exercises over receiving random exercises, indicating that contextual personalization can enhance student engagement.
The study also discusses the challenges and limitations of using large language models for generating personalized exercises, such as the need for careful prompt engineering and the potential for biased or offensive content. Overall, the results suggest that AI-generated programming exercises can be a valuable addition to introductory programming courses, providing students with a wide range of practice material tailored to their interests and needs.This article evaluates the quality and effectiveness of contextually personalized programming exercises generated by GPT-4 in an introductory programming course. The study aims to address the time-consuming and labor-intensive task of creating personalized exercises for students, which is a common challenge for computer science educators. The research is grounded in educational psychology, which suggests that context personalization can enhance students' situational interest and engagement. The study involves both expert and student evaluations of the generated exercises, as well as an analysis of student interactions with the system.
Key findings include:
- **Quality of Exercises**: Both experts and students rated the exercises highly, indicating that GPT-4-generated exercises are generally of good quality.
- **Student Engagement**: Students found the exercises engaging and useful, suggesting that AI-generated exercises can effectively supplement traditional teaching methods.
- **Contextual Personalization**: Students preferred choosing the theme of the exercises over receiving random exercises, indicating that contextual personalization can enhance student engagement.
The study also discusses the challenges and limitations of using large language models for generating personalized exercises, such as the need for careful prompt engineering and the potential for biased or offensive content. Overall, the results suggest that AI-generated programming exercises can be a valuable addition to introductory programming courses, providing students with a wide range of practice material tailored to their interests and needs.