March 18-21, 2024 | Xiaotong (Tone) Xu, Jiayu Yin, Catherine Gu, Jenny Mar, Sydney Zhang, Jane L. E., Steven P. Dow
This paper explores the use of LLM-enhanced templates for idea reflection, aiming to address the challenges of using large language models (LLMs) in design education. The study finds that novice designers often struggle with prompt engineering and critical reflection on LLM responses. To address these challenges, the authors propose Jamplate, a digital whiteboard plugin that integrates LLM capabilities into design templates, enabling structured interaction and critical reflection. Jamplate provides reflective questions and in-situ guidance to help users think critically and improve their ideas. The authors conducted formative studies with 52 students and five instructors, finding that while LLMs can provide useful information, students often lack the skills to critically evaluate and apply this information. The study also highlights the potential of LLM-enhanced templates to foster critical reflection and improve the design process. The authors developed Jamplate as a prototype, integrating LLMs into two common design templates: Five Whys (5Ys) and Competitive Analysis (CA). The system allows users to interact with LLMs in a structured way, providing reflective questions and guidance to help them think critically about their ideas. A preliminary user study with eight participants found that Jamplate's reflective questions and in-situ guidance improved their ability to think critically and improve their ideas. The study also discusses the potential of LLM-enhanced templates to support critical thinking and creative processes. The authors conclude that integrating LLMs into design templates can enhance the design process by providing structured interaction and critical reflection.This paper explores the use of LLM-enhanced templates for idea reflection, aiming to address the challenges of using large language models (LLMs) in design education. The study finds that novice designers often struggle with prompt engineering and critical reflection on LLM responses. To address these challenges, the authors propose Jamplate, a digital whiteboard plugin that integrates LLM capabilities into design templates, enabling structured interaction and critical reflection. Jamplate provides reflective questions and in-situ guidance to help users think critically and improve their ideas. The authors conducted formative studies with 52 students and five instructors, finding that while LLMs can provide useful information, students often lack the skills to critically evaluate and apply this information. The study also highlights the potential of LLM-enhanced templates to foster critical reflection and improve the design process. The authors developed Jamplate as a prototype, integrating LLMs into two common design templates: Five Whys (5Ys) and Competitive Analysis (CA). The system allows users to interact with LLMs in a structured way, providing reflective questions and guidance to help them think critically about their ideas. A preliminary user study with eight participants found that Jamplate's reflective questions and in-situ guidance improved their ability to think critically and improve their ideas. The study also discusses the potential of LLM-enhanced templates to support critical thinking and creative processes. The authors conclude that integrating LLMs into design templates can enhance the design process by providing structured interaction and critical reflection.