11 Mar 2025 | Jaidev Shriram1*, Alex Trevithick1*, Lingjie Liu2, Ravi Ramamoorthi1
**RealmDreamer** is a technique for generating forward-facing 3D scenes from text prompts using 3D Gaussian Splatting (3DGS) and diffusion models. The key innovation is leveraging 2D inpainting diffusion models to provide low-variance supervision for unknown regions during 3D distillation. Additionally, the method incorporates depth diffusion models to improve the geometry of the generated scenes. The technique does not require video or multi-view data and can synthesize high-quality 3D scenes with complex layouts and realistic geometry. A user study shows that RealmDreamer outperforms existing approaches, being preferred by 88-95%. The method is evaluated on a custom dataset and demonstrates superior quality in appearance and geometry compared to baselines. The technique also extends to generating 3D scenes from a single image, filling in occluded areas and generating realistic geometry.**RealmDreamer** is a technique for generating forward-facing 3D scenes from text prompts using 3D Gaussian Splatting (3DGS) and diffusion models. The key innovation is leveraging 2D inpainting diffusion models to provide low-variance supervision for unknown regions during 3D distillation. Additionally, the method incorporates depth diffusion models to improve the geometry of the generated scenes. The technique does not require video or multi-view data and can synthesize high-quality 3D scenes with complex layouts and realistic geometry. A user study shows that RealmDreamer outperforms existing approaches, being preferred by 88-95%. The method is evaluated on a custom dataset and demonstrates superior quality in appearance and geometry compared to baselines. The technique also extends to generating 3D scenes from a single image, filling in occluded areas and generating realistic geometry.