**Octree-GS: Towards Consistent Real-time Rendering with LOD-Structured 3D Gaussians** **Authors:** Kerui Ren, Lihan Jiang, Tao Lu, Mulin Yu, Linning Xu, Zhangkai Ni, Bo Dai **Institution:** Shanghai Artificial Intelligence Laboratory, Tongji University, University of Science and Technology of China, The Chinese University of Hong Kong **Abstract:** Recent advancements in 3D Gaussian splatting (3D-GS) have shown significant improvements in rendering fidelity and efficiency compared to NeRF-based neural scene representations. However, 3D-GS faces challenges in large scenes with complex details due to the excessive number of Gaussian primitives within the viewing frustum, leading to inconsistent rendering speeds and visual artifacts. To address these issues, the authors introduce Octree-GS, a novel approach that incorporates Level-of-Detail (LOD) techniques to manage the complexity of 3D Gaussians. Octree-GS dynamically selects appropriate levels from a set of multi-resolution anchor points, ensuring consistent rendering performance with adaptive LOD adjustments while maintaining high-fidelity results. **Key Contributions:** 1. **Octree Structure:** Octree-GS uses an octree structure to organize 3D Gaussians hierarchically, allowing for efficient management of scene details. 2. **Adaptive LOD Selection:** The model dynamically selects the appropriate LOD based on the viewing distance and scene richness, reducing the number of Gaussian primitives used in rendering. 3. **Progressive Training:** A progressive training strategy is employed to encourage distinct roles of different LODs, stabilizing the training process and enhancing rendering quality. **Methods:** - **LOD-structured 3D Gaussians:** Octree-GS initializes anchor Gaussians using an octree structure and dynamically assigns them to each LOD level. The model adaptively refines anchor points to capture more details and removes redundant points. - **Level Selection:** The model dynamically fetches anchor Gaussians from the appropriate LOD levels based on the viewing distance and scene complexity, ensuring efficient rendering. - **Progressive Training:** A coarse-to-fine optimization strategy is used to prevent overlapping between different LOD levels, enhancing the rendering accuracy of coarser LODs. **Experiments:** - **Dataset and Metrics:** Evaluations are conducted on various datasets, including Mip-NeRF360, Tanks & Temples, DeepBlending, BungeeNeRF, and MatrixCity. - **Results Analysis:** Octree-GS demonstrates superior rendering quality and efficiency compared to baselines, with reduced Gaussian counts and faster rendering speeds in large-scale scenes and extreme-view sequences. **Conclusion:** Octree-GS addresses the limitations of 3D-GS by introducing an LOD-structured approach, enhancing detail capture while maintaining real-time rendering performance. Future work includes further optimizing hyperparameters and addressing challenges such as dependency on initial sparse point clouds and geometry support.**Octree-GS: Towards Consistent Real-time Rendering with LOD-Structured 3D Gaussians** **Authors:** Kerui Ren, Lihan Jiang, Tao Lu, Mulin Yu, Linning Xu, Zhangkai Ni, Bo Dai **Institution:** Shanghai Artificial Intelligence Laboratory, Tongji University, University of Science and Technology of China, The Chinese University of Hong Kong **Abstract:** Recent advancements in 3D Gaussian splatting (3D-GS) have shown significant improvements in rendering fidelity and efficiency compared to NeRF-based neural scene representations. However, 3D-GS faces challenges in large scenes with complex details due to the excessive number of Gaussian primitives within the viewing frustum, leading to inconsistent rendering speeds and visual artifacts. To address these issues, the authors introduce Octree-GS, a novel approach that incorporates Level-of-Detail (LOD) techniques to manage the complexity of 3D Gaussians. Octree-GS dynamically selects appropriate levels from a set of multi-resolution anchor points, ensuring consistent rendering performance with adaptive LOD adjustments while maintaining high-fidelity results. **Key Contributions:** 1. **Octree Structure:** Octree-GS uses an octree structure to organize 3D Gaussians hierarchically, allowing for efficient management of scene details. 2. **Adaptive LOD Selection:** The model dynamically selects the appropriate LOD based on the viewing distance and scene richness, reducing the number of Gaussian primitives used in rendering. 3. **Progressive Training:** A progressive training strategy is employed to encourage distinct roles of different LODs, stabilizing the training process and enhancing rendering quality. **Methods:** - **LOD-structured 3D Gaussians:** Octree-GS initializes anchor Gaussians using an octree structure and dynamically assigns them to each LOD level. The model adaptively refines anchor points to capture more details and removes redundant points. - **Level Selection:** The model dynamically fetches anchor Gaussians from the appropriate LOD levels based on the viewing distance and scene complexity, ensuring efficient rendering. - **Progressive Training:** A coarse-to-fine optimization strategy is used to prevent overlapping between different LOD levels, enhancing the rendering accuracy of coarser LODs. **Experiments:** - **Dataset and Metrics:** Evaluations are conducted on various datasets, including Mip-NeRF360, Tanks & Temples, DeepBlending, BungeeNeRF, and MatrixCity. - **Results Analysis:** Octree-GS demonstrates superior rendering quality and efficiency compared to baselines, with reduced Gaussian counts and faster rendering speeds in large-scale scenes and extreme-view sequences. **Conclusion:** Octree-GS addresses the limitations of 3D-GS by introducing an LOD-structured approach, enhancing detail capture while maintaining real-time rendering performance. Future work includes further optimizing hyperparameters and addressing challenges such as dependency on initial sparse point clouds and geometry support.