19 Jan 2024 | PRIYAN VAITHILINGAM, Harvard University, USA; ELENA L. GLASSMAN, Harvard University, USA; JEEVANA PRIYA INALA, Microsoft, USA; CHENGLONG WANG, Microsoft, USA
**DynaVis: Dynamically Synthesized UI Widgets for Visualization Editing**
DynaVis is a novel approach that combines natural language interfaces (NLIs) with dynamically synthesized user interface (UI) widgets to enhance the editing and refinement of visualizations. The system allows users to describe editing tasks in natural language, which DynaVis then translates into persistent widgets for further modifications. This blend of NLIs and GUI widgets reduces the complexity of traditional GUIs and provides immediate visual feedback, addressing the limitations of both approaches.
**Key Features:**
- **Natural Language Command Input:** Users can describe editing tasks using natural language, such as "rotate x-axis label 45 degrees."
- **Dynamic Widget Synthesis:** DynaVis generates widgets based on user commands, allowing for fine-grained edits and immediate visual feedback.
- **Modular Design:** Widgets are designed to be modular, ensuring that edits from different widgets do not conflict with each other.
- **Large Language Model (LLM) Integration:** The system leverages LLMs to translate natural language inputs into HTML scripts and JavaScript callback functions, enabling dynamic widget synthesis.
- **User Study:** A study with 24 participants showed that DynaVis significantly reduced the effort required to complete visualization editing tasks compared to a baseline NL-based tool, with users preferring DynaVis due to its ease of use and immediate visual feedback.
**System Design:**
- **Data Summarization:** A data summarizer generates a compact summary of the dataset to provide context for LLMs.
- **LLM-Based Synthesis:** The Chart Engine and Widget Engine use LLMs to synthesize visualizations and widgets based on user inputs.
- **Post-Processing:** Post-processing steps ensure the validity and responsiveness of the synthesized widgets.
**User Study:**
- **Task Completion:** Participants using DynaVis completed tasks more successfully and with less effort compared to those using the baseline tool.
- **Cognitive Load:** Participants reported lower mental demand, less hurry, more success, less effort, and less frustration when using DynaVis.
- **User Behavior:** Dynamic widgets were used more frequently, reducing the need for natural language commands and allowing for easier exploration and fine-tuning of edits.
- **User Preference:** All but one participant strongly preferred DynaVis over the baseline tool, citing ease of repetitive edits and enhanced understanding through visual feedback.
**Conclusion:**
DynaVis addresses the challenges of visualization editing by combining the benefits of NLIs and GUI widgets, providing a more intuitive and efficient way to refine visualizations.**DynaVis: Dynamically Synthesized UI Widgets for Visualization Editing**
DynaVis is a novel approach that combines natural language interfaces (NLIs) with dynamically synthesized user interface (UI) widgets to enhance the editing and refinement of visualizations. The system allows users to describe editing tasks in natural language, which DynaVis then translates into persistent widgets for further modifications. This blend of NLIs and GUI widgets reduces the complexity of traditional GUIs and provides immediate visual feedback, addressing the limitations of both approaches.
**Key Features:**
- **Natural Language Command Input:** Users can describe editing tasks using natural language, such as "rotate x-axis label 45 degrees."
- **Dynamic Widget Synthesis:** DynaVis generates widgets based on user commands, allowing for fine-grained edits and immediate visual feedback.
- **Modular Design:** Widgets are designed to be modular, ensuring that edits from different widgets do not conflict with each other.
- **Large Language Model (LLM) Integration:** The system leverages LLMs to translate natural language inputs into HTML scripts and JavaScript callback functions, enabling dynamic widget synthesis.
- **User Study:** A study with 24 participants showed that DynaVis significantly reduced the effort required to complete visualization editing tasks compared to a baseline NL-based tool, with users preferring DynaVis due to its ease of use and immediate visual feedback.
**System Design:**
- **Data Summarization:** A data summarizer generates a compact summary of the dataset to provide context for LLMs.
- **LLM-Based Synthesis:** The Chart Engine and Widget Engine use LLMs to synthesize visualizations and widgets based on user inputs.
- **Post-Processing:** Post-processing steps ensure the validity and responsiveness of the synthesized widgets.
**User Study:**
- **Task Completion:** Participants using DynaVis completed tasks more successfully and with less effort compared to those using the baseline tool.
- **Cognitive Load:** Participants reported lower mental demand, less hurry, more success, less effort, and less frustration when using DynaVis.
- **User Behavior:** Dynamic widgets were used more frequently, reducing the need for natural language commands and allowing for easier exploration and fine-tuning of edits.
- **User Preference:** All but one participant strongly preferred DynaVis over the baseline tool, citing ease of repetitive edits and enhanced understanding through visual feedback.
**Conclusion:**
DynaVis addresses the challenges of visualization editing by combining the benefits of NLIs and GUI widgets, providing a more intuitive and efficient way to refine visualizations.