A Hierarchical 3D Gaussian Representation for Real-Time Rendering of Very Large Datasets

A Hierarchical 3D Gaussian Representation for Real-Time Rendering of Very Large Datasets

July 2024 | BERNHARD KERBL, ANDREAS MEULEMAN, GEORGIOS KOPANAS, MICHAEL WIMMER, ALEXANDRE LANVIN, GEORGE DRETTAKIS
A Hierarchical 3D Gaussian Representation for Real-Time Rendering of Very Large Datasets This paper introduces a hierarchical 3D Gaussian representation for real-time rendering of very large datasets. The method allows efficient rendering of massive data by introducing a 3D Gaussian Splatting hierarchy that preserves visual quality for large scenes while offering an efficient Level-of-Detail (LOD) solution for rendering distant content. The method uses a divide-and-conquer approach to train very large scenes in independent chunks, and consolidates the chunks into a hierarchy that can be optimized to improve visual quality. The method also adapts training to account for sparse coverage of the scene, which is common in large captures. The method enables real-time rendering of very large scenes and can adapt to available resources thanks to the LOD method. The method is demonstrated on several datasets, including one provided by Wayne and three captured using a bicycle helmet-mounted rig with 5 or 6 GoPro cameras. The datasets cover distances from 450m up to several kilometers, with 5,800 to 28,000 images. The method allows real-time navigation in 3D. The contributions include a new hierarchy for 3DGS that allows efficient level selection and interpolation, a method to optimize the interior nodes of the hierarchy to improve visual quality, and chunk-based divide-and-conquer training and rendering for large scenes. The method enables parallel training of chunks of very large scenes and is the first solution with full dynamic LOD, allowing real-time rendering of radiance fields for scenes of such size. The method adapts to available resources and can be used with cheap, consumer-level equipment for capture, making capturing and rendering neighborhood-scale scenes accessible to anyone. The source code, including all supporting code for capture and calibration, will be released.A Hierarchical 3D Gaussian Representation for Real-Time Rendering of Very Large Datasets This paper introduces a hierarchical 3D Gaussian representation for real-time rendering of very large datasets. The method allows efficient rendering of massive data by introducing a 3D Gaussian Splatting hierarchy that preserves visual quality for large scenes while offering an efficient Level-of-Detail (LOD) solution for rendering distant content. The method uses a divide-and-conquer approach to train very large scenes in independent chunks, and consolidates the chunks into a hierarchy that can be optimized to improve visual quality. The method also adapts training to account for sparse coverage of the scene, which is common in large captures. The method enables real-time rendering of very large scenes and can adapt to available resources thanks to the LOD method. The method is demonstrated on several datasets, including one provided by Wayne and three captured using a bicycle helmet-mounted rig with 5 or 6 GoPro cameras. The datasets cover distances from 450m up to several kilometers, with 5,800 to 28,000 images. The method allows real-time navigation in 3D. The contributions include a new hierarchy for 3DGS that allows efficient level selection and interpolation, a method to optimize the interior nodes of the hierarchy to improve visual quality, and chunk-based divide-and-conquer training and rendering for large scenes. The method enables parallel training of chunks of very large scenes and is the first solution with full dynamic LOD, allowing real-time rendering of radiance fields for scenes of such size. The method adapts to available resources and can be used with cheap, consumer-level equipment for capture, making capturing and rendering neighborhood-scale scenes accessible to anyone. The source code, including all supporting code for capture and calibration, will be released.
Reach us at info@study.space