Shaping Human-AI Collaboration: Varied Scaffolding Levels in Co-writing with Language Models

Shaping Human-AI Collaboration: Varied Scaffolding Levels in Co-writing with Language Models

May 11–16, 2024 | Paramveer S. Dhillon, Somayeh Molaei, Jiaqi Li, Maximilian Golub, Shaochun Zheng, Lionel P. Robert
This paper explores how varying levels of scaffolding from large language models (LLMs) impact the co-writing process between humans and AI. Through a within-subjects field experiment with 131 participants, the study examines the effects of no AI assistance, next-sentence suggestions (low scaffolding), and next-paragraph suggestions (high scaffolding) on writing quality and productivity. The findings reveal a U-shaped relationship: low scaffolding did not significantly improve writing quality or productivity, while high scaffolding led to significant improvements, particularly benefiting non-regular writers and less tech-savvy users. However, high scaffolding also resulted in a moderate decrease in text ownership and satisfaction. The study highlights the need for personalized scaffolding mechanisms in AI-powered writing tools to enhance both productivity and user satisfaction. The research contributes to the understanding of how different levels of AI input influence user experience and writing outcomes, providing practical guidelines for designing AI-assisted writing tools that are user-centric and maintain human agency.This paper explores how varying levels of scaffolding from large language models (LLMs) impact the co-writing process between humans and AI. Through a within-subjects field experiment with 131 participants, the study examines the effects of no AI assistance, next-sentence suggestions (low scaffolding), and next-paragraph suggestions (high scaffolding) on writing quality and productivity. The findings reveal a U-shaped relationship: low scaffolding did not significantly improve writing quality or productivity, while high scaffolding led to significant improvements, particularly benefiting non-regular writers and less tech-savvy users. However, high scaffolding also resulted in a moderate decrease in text ownership and satisfaction. The study highlights the need for personalized scaffolding mechanisms in AI-powered writing tools to enhance both productivity and user satisfaction. The research contributes to the understanding of how different levels of AI input influence user experience and writing outcomes, providing practical guidelines for designing AI-assisted writing tools that are user-centric and maintain human agency.
Reach us at info@study.space
[slides and audio] Shaping Human-AI Collaboration%3A Varied Scaffolding Levels in Co-writing with Language Models