14 Jun 2024 | Hong-Xing Yu, Haoyi Duan, Charles Herrmann, William T. Freeman, Jiajun Wu
WonderWorld is an interactive 3D scene generation framework that allows users to explore and shape virtual environments based on a single input image and text prompts. The system generates connected and diverse 3D scenes in less than 10 seconds on a single A6000 GPU, enabling real-time user interaction and exploration. It addresses the challenges of scene generation speed and geometric distortion by leveraging Fast Gaussian Surfels and a guided diffusion-based depth estimation method. The framework generates geometrically consistent extrapolation while significantly reducing computational time, making it suitable for applications in virtual reality, gaming, and creative design. The system allows users to interactively generate and navigate immersive, potentially infinite virtual worlds from a single image. The approach represents a significant advancement in interactive 3D scene generation, opening up new possibilities for user-driven content creation and exploration in virtual environments. The full code and software for reproducibility are available at the project website: https://WonderWorld-2024.github.io/.WonderWorld is an interactive 3D scene generation framework that allows users to explore and shape virtual environments based on a single input image and text prompts. The system generates connected and diverse 3D scenes in less than 10 seconds on a single A6000 GPU, enabling real-time user interaction and exploration. It addresses the challenges of scene generation speed and geometric distortion by leveraging Fast Gaussian Surfels and a guided diffusion-based depth estimation method. The framework generates geometrically consistent extrapolation while significantly reducing computational time, making it suitable for applications in virtual reality, gaming, and creative design. The system allows users to interactively generate and navigate immersive, potentially infinite virtual worlds from a single image. The approach represents a significant advancement in interactive 3D scene generation, opening up new possibilities for user-driven content creation and exploration in virtual environments. The full code and software for reproducibility are available at the project website: https://WonderWorld-2024.github.io/.