BrightDreamer: Generic 3D Gaussian Generative Framework for Fast Text-to-3D Synthesis

BrightDreamer: Generic 3D Gaussian Generative Framework for Fast Text-to-3D Synthesis

18 Nov 2024 | Lutao Jiang, Xu Zheng, Yuanhuiyi Lyu, Jiazhou Zhou, Lin Wang
BrightDreamer is a novel end-to-end feed-forward framework for fast text-to-3D synthesis, achieving a generation speed of 77 milliseconds. The framework addresses the inefficiency of existing per-prompt optimization methods by directly generating 3D Gaussians from text prompts without iterative refinement. The key idea is to deform predefined anchor positions to estimate the centers of 3D Gaussians, guided by text prompts. A Text-guided Shape Deformation (TSD) network is used to predict the deformed shape and new positions, which serve as the centers of the 3D Gaussians. Additionally, a Text-guided Triplane Generator (TTG) is designed to generate a triplane representation for the 3D object, which is then decoded into the other four attributes of the 3D Gaussians (scaling, rotation, opacity, and SH coefficients). The generated 3D Gaussians can be rendered at 705 frames per second. BrightDreamer demonstrates strong semantic understanding and generalization capabilities, even for complex text prompts. The framework is trained on a diverse set of text prompts and can generate 3D content from any unseen text prompt. The code is available on the project page. Extensive experiments show that BrightDreamer outperforms existing methods in terms of generation speed and quality. The framework is also capable of interpolating between two prompts, allowing for creative design exploration. The method leverages the efficiency of 2D image diffusion models to train 3D generative models, which is more resource-efficient than traditional methods. The framework is designed to be generic, enabling fast and efficient text-to-3D synthesis. The results show that BrightDreamer can generate high-quality 3D content from text prompts, even for unseen words, demonstrating its strong generalization ability. The framework is compared with other methods, including per-prompt text-to-3DGS, amortized text-to-NeRF, and text-to-image-to-3DGS methods, and shows superior performance in terms of generation speed and quality. The method is also capable of fine-tuning the generated 3D Gaussians to enhance their quality. The framework is trained on a diverse set of text prompts and can generate 3D content from any unseen text prompt. The code is available on the project page. The results show that BrightDreamer can generate high-quality 3D content from text prompts, even for unseen words, demonstrating its strong generalization ability. The framework is compared with other methods, including per-prompt text-to-3DGS, amortized text-to-NeRF, and text-to-image-to-3DGS methods, and shows superior performance in terms of generation speed and quality. The method is also capable of fine-tuning the generated 3D Gaussians to enhance their quality.BrightDreamer is a novel end-to-end feed-forward framework for fast text-to-3D synthesis, achieving a generation speed of 77 milliseconds. The framework addresses the inefficiency of existing per-prompt optimization methods by directly generating 3D Gaussians from text prompts without iterative refinement. The key idea is to deform predefined anchor positions to estimate the centers of 3D Gaussians, guided by text prompts. A Text-guided Shape Deformation (TSD) network is used to predict the deformed shape and new positions, which serve as the centers of the 3D Gaussians. Additionally, a Text-guided Triplane Generator (TTG) is designed to generate a triplane representation for the 3D object, which is then decoded into the other four attributes of the 3D Gaussians (scaling, rotation, opacity, and SH coefficients). The generated 3D Gaussians can be rendered at 705 frames per second. BrightDreamer demonstrates strong semantic understanding and generalization capabilities, even for complex text prompts. The framework is trained on a diverse set of text prompts and can generate 3D content from any unseen text prompt. The code is available on the project page. Extensive experiments show that BrightDreamer outperforms existing methods in terms of generation speed and quality. The framework is also capable of interpolating between two prompts, allowing for creative design exploration. The method leverages the efficiency of 2D image diffusion models to train 3D generative models, which is more resource-efficient than traditional methods. The framework is designed to be generic, enabling fast and efficient text-to-3D synthesis. The results show that BrightDreamer can generate high-quality 3D content from text prompts, even for unseen words, demonstrating its strong generalization ability. The framework is compared with other methods, including per-prompt text-to-3DGS, amortized text-to-NeRF, and text-to-image-to-3DGS methods, and shows superior performance in terms of generation speed and quality. The method is also capable of fine-tuning the generated 3D Gaussians to enhance their quality. The framework is trained on a diverse set of text prompts and can generate 3D content from any unseen text prompt. The code is available on the project page. The results show that BrightDreamer can generate high-quality 3D content from text prompts, even for unseen words, demonstrating its strong generalization ability. The framework is compared with other methods, including per-prompt text-to-3DGS, amortized text-to-NeRF, and text-to-image-to-3DGS methods, and shows superior performance in terms of generation speed and quality. The method is also capable of fine-tuning the generated 3D Gaussians to enhance their quality.
Reach us at info@study.space