9 Apr 2024 | Samuel Rota Bulò, Lorenzo Porzi, and Peter Kontschieder
This paper addresses the limitations of Adaptive Density Control (ADC) in 3D Gaussian Splatting (3DGS), a method for high-quality, photorealistic scene representation. The main contributions include:
1. **Pixel-Error Driven Formulation**: Introduces a more principled, pixel-error driven formulation for density control in 3DGS, leveraging an auxiliary, per-pixel error function as the criterion for densification.
2. **Error-Based Densification**: Proposes a novel decision criterion for densification based on per-pixel errors, which are propagated back to contributing Gaussian primitives. This approach is more intuitive and robust compared to the gradient-based approach used in ADC.
3. **Opacity Correction**: Corrects a bias in the current opacity handling strategy of ADC during cloning operations, which can lead to an overall increase in opacity in the cloned region, affecting the densification process.
4. **Primitives Growth Control**: Introduces a mechanism to control the total number of primitives generated per scene and the maximum amount of novel primitives introduced per densification run, avoiding out-of-memory errors and better tuning the method's behavior.
The paper evaluates the effectiveness of these improvements on standard 3DGS and its Mip-Splatting extension using benchmark datasets like Mip-NeRF 360, Tanks and Temples, and Deep Blending. The results show consistent and systematic improvements over previous methods, particularly in perceptual metrics like LPIPS. The proposed approach maintains the same quality as 3DGS in non-problematic areas while effectively addressing under-fitting issues in challenging scenes.This paper addresses the limitations of Adaptive Density Control (ADC) in 3D Gaussian Splatting (3DGS), a method for high-quality, photorealistic scene representation. The main contributions include:
1. **Pixel-Error Driven Formulation**: Introduces a more principled, pixel-error driven formulation for density control in 3DGS, leveraging an auxiliary, per-pixel error function as the criterion for densification.
2. **Error-Based Densification**: Proposes a novel decision criterion for densification based on per-pixel errors, which are propagated back to contributing Gaussian primitives. This approach is more intuitive and robust compared to the gradient-based approach used in ADC.
3. **Opacity Correction**: Corrects a bias in the current opacity handling strategy of ADC during cloning operations, which can lead to an overall increase in opacity in the cloned region, affecting the densification process.
4. **Primitives Growth Control**: Introduces a mechanism to control the total number of primitives generated per scene and the maximum amount of novel primitives introduced per densification run, avoiding out-of-memory errors and better tuning the method's behavior.
The paper evaluates the effectiveness of these improvements on standard 3DGS and its Mip-Splatting extension using benchmark datasets like Mip-NeRF 360, Tanks and Temples, and Deep Blending. The results show consistent and systematic improvements over previous methods, particularly in perceptual metrics like LPIPS. The proposed approach maintains the same quality as 3DGS in non-problematic areas while effectively addressing under-fitting issues in challenging scenes.