August 2023 | BERNHARD KERBL*, Inria, Université Côte d'Azur, France GEORGIOS KOPANAS*, Inria, Université Côte d'Azur, France THOMAS LEIMKÜHLER, Max-Planck-Institut für Informatik, Germany GEORGE DRETTAKIS, Inria, Université Côte d'Azur, France
The paper introduces a novel method for real-time rendering of radiance fields, achieving state-of-the-art visual quality while maintaining competitive training times. The key contributions are:
1. **3D Gaussian Representation**: The method uses 3D Gaussians to represent the scene, which are differentiable and can be efficiently rasterized. This representation allows for flexible optimization and fast rendering.
2. **Optimization with Adaptive Density Control**: The 3D Gaussians are optimized in an interleaved manner with adaptive density control, ensuring accurate and compact representations of the scene. This process includes creating, moving, and removing Gaussians to better fit the geometry.
3. **Fast Visibility-Aware Rendering Algorithm**: A tile-based rasterizer is developed to support anisotropic splatting and visibility ordering, enabling fast and accurate rendering. The algorithm also allows for efficient backpropagation through the entire scene.
The method is evaluated on several datasets, showing that it achieves high-quality results comparable to or better than existing methods, such as Mip-NeRF360, while requiring significantly less training time. The paper also discusses the limitations and future work, including the need for regularization and memory optimization techniques.The paper introduces a novel method for real-time rendering of radiance fields, achieving state-of-the-art visual quality while maintaining competitive training times. The key contributions are:
1. **3D Gaussian Representation**: The method uses 3D Gaussians to represent the scene, which are differentiable and can be efficiently rasterized. This representation allows for flexible optimization and fast rendering.
2. **Optimization with Adaptive Density Control**: The 3D Gaussians are optimized in an interleaved manner with adaptive density control, ensuring accurate and compact representations of the scene. This process includes creating, moving, and removing Gaussians to better fit the geometry.
3. **Fast Visibility-Aware Rendering Algorithm**: A tile-based rasterizer is developed to support anisotropic splatting and visibility ordering, enabling fast and accurate rendering. The algorithm also allows for efficient backpropagation through the entire scene.
The method is evaluated on several datasets, showing that it achieves high-quality results comparable to or better than existing methods, such as Mip-NeRF360, while requiring significantly less training time. The paper also discusses the limitations and future work, including the need for regularization and memory optimization techniques.