Towards Responsible Development of Generative AI for Education: An Evaluation-Driven Approach

Towards Responsible Development of Generative AI for Education: An Evaluation-Driven Approach

2024-05-14 | Irina Jurenska, Markus Kunesch, Kevin R. McKee, Daniel Gillick, Shaojian Zhu, Sara Wilberger, Shubham Milind Phal, Katherine Hermann, Daniel Kasenberg, Avishkar Bhoopchand, Ankit Anand, Miruna Pislar, Stephanie Chan, Lisa Wang, Jennifer She, Parsa Mahmoudieh, Aliya Rysbek, Wei-Jen Ko, Andrea Huber, Brett Wiltshire, Gal Eldan, Roni Rabin, Jasmin Rubinovitz, Amit Piturac, Mac McAllister, Julia Wilkowski, David Choi, Roe Engelberg, Lidan Hackmon, Adva Levin, Rachel Griffin, Michael Sears, Filip Bar, Mia Mesar, Mana Jabbour, Arslan Chaudhry, James Cohan, Sridhar Thiagarajan, Nir Levine, Ben Brown, Dilan Gorur, Svetlana Grant, Rachel Hashimshoni, Laura Weidinger, Jieru Hu, Dawn Chen, Kuba Dolecki, Canfer Akbulut, Maxwell Bileschi, Laura Culpa, Wen-Xin Dong, Nahema Marchal, Kelsie Van Deman, Hema Bajaj Misra, Michael Duah, Moran Ambar, Avi Caciularu, Sandra Lefdal, Chris Summerfield, James An, Pierre-Alexandre Kamienny, Abhinit Mohd, Theofilos Strinopoulous, Annie Hale, Wayne Anderson, Luis C. Cobo, Niv Efron, Muktha Ananda, Shakir Mohamed, Maureen Heymans, Zoubin Ghahramani, Yossi Matias, Ben Gomes, and Lila Ibrahim
The paper "Towards Responsible Development of Generative AI for Education: An Evaluation-Driven Approach" addresses the challenge of providing equitable and universal access to quality education through the development of generative AI (gen AI) tools. The authors highlight the potential of gen AI to offer personalized tutoring and teaching assistance, but note that this potential has not yet been fully realized due to difficulties in verbalizing pedagogical intuitions and the lack of good evaluation practices. They present a participatory and multidisciplinary approach, involving learners, educators, policymakers, and researchers, to translate high-level principles from learning science into practical educational benchmarks. The paper introduces LearnLM-Tutor, a text-based gen AI tutor fine-tuned for 1:1 conversational tutoring, and demonstrates its superior performance over a prompt-tuned version of Gemini 1.0 in various pedagogical dimensions. The authors also develop a comprehensive suite of seven pedagogical benchmarks, including quantitative, qualitative, automatic, and human evaluations, to assess the performance of conversational AI tutors. The paper discusses the limitations, safety, ethical, and policy implications of their work, emphasizing the need for a shared framework across learning science, EdTech, and AI for Education to guide the responsible development of gen AI in education.The paper "Towards Responsible Development of Generative AI for Education: An Evaluation-Driven Approach" addresses the challenge of providing equitable and universal access to quality education through the development of generative AI (gen AI) tools. The authors highlight the potential of gen AI to offer personalized tutoring and teaching assistance, but note that this potential has not yet been fully realized due to difficulties in verbalizing pedagogical intuitions and the lack of good evaluation practices. They present a participatory and multidisciplinary approach, involving learners, educators, policymakers, and researchers, to translate high-level principles from learning science into practical educational benchmarks. The paper introduces LearnLM-Tutor, a text-based gen AI tutor fine-tuned for 1:1 conversational tutoring, and demonstrates its superior performance over a prompt-tuned version of Gemini 1.0 in various pedagogical dimensions. The authors also develop a comprehensive suite of seven pedagogical benchmarks, including quantitative, qualitative, automatic, and human evaluations, to assess the performance of conversational AI tutors. The paper discusses the limitations, safety, ethical, and policy implications of their work, emphasizing the need for a shared framework across learning science, EdTech, and AI for Education to guide the responsible development of gen AI in education.
Reach us at info@study.space