April 30, 2024 | Yilin Ye, Jianing Hao, Yihan Hou, Zhan Wang, Shishi Xiao, Yuyu Luo, Wei Zeng
Generative AI (GenAI) has made significant progress in recent years, showing impressive performance in various generation tasks across domains like computer vision and computational design. Researchers have integrated GenAI into visualization frameworks, leveraging its superior generative capacity for different operations. Recent breakthroughs in GenAI, such as diffusion models and large language models, have greatly enhanced the potential of GenAI for visualization (GenAI4VIS). This paper reviews previous studies that have used GenAI for visualization and discusses challenges and opportunities for future research. It covers the applications of different GenAI methods, including sequence, tabular, spatial, and graph generation techniques, for various visualization tasks, summarizing them into four major stages: data enhancement, visual mapping generation, stylization, and interaction. For each specific visualization sub-task, the paper illustrates typical data and concrete GenAI algorithms to provide an in-depth understanding of the state-of-the-art GenAI4VIS techniques and their limitations. It also discusses three major aspects of challenges and research opportunities: evaluation, dataset, and the gap between end-to-end GenAI and generative algorithms. By summarizing different generation algorithms, their current applications, and limitations, this paper aims to provide useful insights for future GenAI4VIS research. The paper is structured as follows: Section 2 outlines the scope and taxonomy of the survey with definitions of key concepts. Sections 3-6 each correspond to a stage in the visualization pipeline where GenAI has been used. Section 3 concerns the use of GenAI for data enhancement. Section 4 summarizes works leveraging GenAI for visual mapping generation. Section 5 focuses on how GenAI is utilized for stylization and communication with visualization. Section 6 covers GenAI techniques to support user interaction. Each subsection in Sections 3-6 covers a specific task in the stage. Instead of listing the works one by one, the structure of the subsection is divided into two parts: data & algorithm and discussion, for a comprehensive understanding of how the current GenAI method works for data of certain structures and what remains challenging for GenAI in particular tasks. Finally, Section 7 discusses some dominant challenges and research opportunities for future research. The paper provides a comprehensive overview of the state-of-the-art GenAI4VIS methods, highlighting their applications, challenges, and opportunities for future research.Generative AI (GenAI) has made significant progress in recent years, showing impressive performance in various generation tasks across domains like computer vision and computational design. Researchers have integrated GenAI into visualization frameworks, leveraging its superior generative capacity for different operations. Recent breakthroughs in GenAI, such as diffusion models and large language models, have greatly enhanced the potential of GenAI for visualization (GenAI4VIS). This paper reviews previous studies that have used GenAI for visualization and discusses challenges and opportunities for future research. It covers the applications of different GenAI methods, including sequence, tabular, spatial, and graph generation techniques, for various visualization tasks, summarizing them into four major stages: data enhancement, visual mapping generation, stylization, and interaction. For each specific visualization sub-task, the paper illustrates typical data and concrete GenAI algorithms to provide an in-depth understanding of the state-of-the-art GenAI4VIS techniques and their limitations. It also discusses three major aspects of challenges and research opportunities: evaluation, dataset, and the gap between end-to-end GenAI and generative algorithms. By summarizing different generation algorithms, their current applications, and limitations, this paper aims to provide useful insights for future GenAI4VIS research. The paper is structured as follows: Section 2 outlines the scope and taxonomy of the survey with definitions of key concepts. Sections 3-6 each correspond to a stage in the visualization pipeline where GenAI has been used. Section 3 concerns the use of GenAI for data enhancement. Section 4 summarizes works leveraging GenAI for visual mapping generation. Section 5 focuses on how GenAI is utilized for stylization and communication with visualization. Section 6 covers GenAI techniques to support user interaction. Each subsection in Sections 3-6 covers a specific task in the stage. Instead of listing the works one by one, the structure of the subsection is divided into two parts: data & algorithm and discussion, for a comprehensive understanding of how the current GenAI method works for data of certain structures and what remains challenging for GenAI in particular tasks. Finally, Section 7 discusses some dominant challenges and research opportunities for future research. The paper provides a comprehensive overview of the state-of-the-art GenAI4VIS methods, highlighting their applications, challenges, and opportunities for future research.