GALA3D: Towards Text-to-3D Complex Scene Generation via Layout-guided Generative Gaussian Splatting

GALA3D: Towards Text-to-3D Complex Scene Generation via Layout-guided Generative Gaussian Splatting

11 Jun 2024 | Xiaoyu Zhou 1 Xingjian Ran 1 Yajiao Xiong 1 Jinlin He 1 Zhiwei Lin 1 Yongtao Wang 1 2 Deqing Sun 3 Ming-Hsuan Yang 3 4
GALA3D is a novel framework for generating complex 3D scenes from textual descriptions using layout-guided generative Gaussian splatting. The method leverages large language models (LLMs) to extract instance relationships and generate coarse layouts, which are then refined to align with the generated 3D scenes. GALA3D introduces a layout-guided Gaussian representation with adaptive geometric control to ensure high-quality geometry and texture. A compositional optimization strategy with diffusion priors is employed to collaboratively generate realistic 3D scenes with consistent geometry, texture, scale, and accurate interactions among multiple objects. The framework supports interactive and controllable editing, making it user-friendly for generating and editing complex 3D content. Experiments demonstrate that GALA3D outperforms existing methods in generating high-fidelity, coherent, and complex 3D scenes with multiple interacting objects.GALA3D is a novel framework for generating complex 3D scenes from textual descriptions using layout-guided generative Gaussian splatting. The method leverages large language models (LLMs) to extract instance relationships and generate coarse layouts, which are then refined to align with the generated 3D scenes. GALA3D introduces a layout-guided Gaussian representation with adaptive geometric control to ensure high-quality geometry and texture. A compositional optimization strategy with diffusion priors is employed to collaboratively generate realistic 3D scenes with consistent geometry, texture, scale, and accurate interactions among multiple objects. The framework supports interactive and controllable editing, making it user-friendly for generating and editing complex 3D content. Experiments demonstrate that GALA3D outperforms existing methods in generating high-fidelity, coherent, and complex 3D scenes with multiple interacting objects.
Reach us at info@study.space