AGG: Amortized Generative 3D Gaussians for Single Image to 3D

AGG: Amortized Generative 3D Gaussians for Single Image to 3D

8 Jan 2024 | Dejia Xu¹, Ye Yuan², Morteza Mardani², Sifei Liu², Jiaming Song², Zhangyang Wang¹, Arash Vahdat²
AGG is an amortized generative 3D Gaussian framework that produces 3D Gaussians from a single image without per-instance optimization. The framework uses a cascaded generation pipeline, starting with a hybrid generator that produces a coarse 3D Gaussian representation and then a super-resolution module that refines it. The hybrid representation allows joint optimization of geometry and texture, while the super-resolution module enhances the resolution and fidelity of the 3D Gaussians. AGG outperforms existing optimization-based and sampling-based methods in both qualitative and quantitative aspects, achieving significantly faster inference speeds. The framework addresses challenges in amortized training by using fixed numbers of Gaussians, canonical isotropic scales, and proper initialization. It also incorporates RGB information to improve texture details during super-resolution. AGG is evaluated on the OmniObject3D dataset, demonstrating competitive performance in generating high-quality 3D content from single images. The method enables zero-shot image-to-object generation and is suitable for real-time applications due to its efficiency.AGG is an amortized generative 3D Gaussian framework that produces 3D Gaussians from a single image without per-instance optimization. The framework uses a cascaded generation pipeline, starting with a hybrid generator that produces a coarse 3D Gaussian representation and then a super-resolution module that refines it. The hybrid representation allows joint optimization of geometry and texture, while the super-resolution module enhances the resolution and fidelity of the 3D Gaussians. AGG outperforms existing optimization-based and sampling-based methods in both qualitative and quantitative aspects, achieving significantly faster inference speeds. The framework addresses challenges in amortized training by using fixed numbers of Gaussians, canonical isotropic scales, and proper initialization. It also incorporates RGB information to improve texture details during super-resolution. AGG is evaluated on the OmniObject3D dataset, demonstrating competitive performance in generating high-quality 3D content from single images. The method enables zero-shot image-to-object generation and is suitable for real-time applications due to its efficiency.
Reach us at info@study.space
[slides] AGG%3A Amortized Generative 3D Gaussians for Single Image to 3D | StudySpace