**RAMBLER: Supporting Writing With Speech via LLM-Assisted Gist Manipulation** **Abstract:** This paper introduces RAMBLER, a novel graphical user interface (GUI) that supports writing with speech by leveraging large language models (LLMs) for gist-level manipulation of dictated text. RAMBLER addresses the challenges of speech production and memory retention by providing two main functions: gist extraction and macro revision. Gist extraction generates keywords and summaries to aid in reviewing and interacting with spoken text, while LLM-assisted macro revisions allow users to respeak, split, merge, and transform dictated text without specifying precise editing locations. A comparative study with 12 participants performing verbal composition tasks showed that RAMBLER outperformed a baseline of a speech-to-text editor plus ChatGPT, facilitating iterative revisions and enhancing user control over content. **Key Contributions:** 1. **RAMBLER Design:** A novel tool that implements a gist-based interface for writing with speech on mobile devices. 2. **Empirical Findings:** Users preferred RAMBLER over the baseline, demonstrating its effectiveness in supporting the development and review of spoken content. 3. **Design Insights:** Insights for designing LLM-supported direct manipulation interfaces. **Related Work:** - **Voice Dictation and Editing:** Research on reducing editing effort and improving speech-to-text accuracy. - **Writing with LLMs:** Studies on the benefits and limitations of LLMs in writing assistance. - **Semantic Manipulations on Text:** Techniques for structural revision and semantic zooming. **RAMBLER Interface Design:** - **Rambles as Units of Interaction:** Users can create and manipulate Rambles, which correspond to single trains of thought. - **Text Reviewing Features:** Semantic Zoom, transcript cleaning, and keyword highlighting to aid in reviewing and navigating spoken text. - **Macro Revision Operations:** Respeaking, reordering, merging, splitting, and transforming Rambles with LLM assistance. **Evaluation:** - **Participants and Tasks:** 12 participants completed two long-form writing tasks using RAMBLER and a baseline interface. - **Data Collection and Analysis:** Screen recordings, audio recordings, and application logs were analyzed for text output quality and user behavior. - **Results:** Text output quality was comparable between RAMBLER and the baseline, and participants found RAMBLER useful for organizing and revising their compositions. **Conclusion:** RAMBLER effectively supports writing with speech by providing a user-friendly interface that leverages LLMs for efficient text manipulation. The study highlights the potential of LLMs in enhancing the writing process and the importance of designing interfaces that facilitate iterative revisions and user control.**RAMBLER: Supporting Writing With Speech via LLM-Assisted Gist Manipulation** **Abstract:** This paper introduces RAMBLER, a novel graphical user interface (GUI) that supports writing with speech by leveraging large language models (LLMs) for gist-level manipulation of dictated text. RAMBLER addresses the challenges of speech production and memory retention by providing two main functions: gist extraction and macro revision. Gist extraction generates keywords and summaries to aid in reviewing and interacting with spoken text, while LLM-assisted macro revisions allow users to respeak, split, merge, and transform dictated text without specifying precise editing locations. A comparative study with 12 participants performing verbal composition tasks showed that RAMBLER outperformed a baseline of a speech-to-text editor plus ChatGPT, facilitating iterative revisions and enhancing user control over content. **Key Contributions:** 1. **RAMBLER Design:** A novel tool that implements a gist-based interface for writing with speech on mobile devices. 2. **Empirical Findings:** Users preferred RAMBLER over the baseline, demonstrating its effectiveness in supporting the development and review of spoken content. 3. **Design Insights:** Insights for designing LLM-supported direct manipulation interfaces. **Related Work:** - **Voice Dictation and Editing:** Research on reducing editing effort and improving speech-to-text accuracy. - **Writing with LLMs:** Studies on the benefits and limitations of LLMs in writing assistance. - **Semantic Manipulations on Text:** Techniques for structural revision and semantic zooming. **RAMBLER Interface Design:** - **Rambles as Units of Interaction:** Users can create and manipulate Rambles, which correspond to single trains of thought. - **Text Reviewing Features:** Semantic Zoom, transcript cleaning, and keyword highlighting to aid in reviewing and navigating spoken text. - **Macro Revision Operations:** Respeaking, reordering, merging, splitting, and transforming Rambles with LLM assistance. **Evaluation:** - **Participants and Tasks:** 12 participants completed two long-form writing tasks using RAMBLER and a baseline interface. - **Data Collection and Analysis:** Screen recordings, audio recordings, and application logs were analyzed for text output quality and user behavior. - **Results:** Text output quality was comparable between RAMBLER and the baseline, and participants found RAMBLER useful for organizing and revising their compositions. **Conclusion:** RAMBLER effectively supports writing with speech by providing a user-friendly interface that leverages LLMs for efficient text manipulation. The study highlights the potential of LLMs in enhancing the writing process and the importance of designing interfaces that facilitate iterative revisions and user control.