5 Apr 2024 | François Darmon, Lorenzo Porzi, Samuel Rota-Bulò, Peter Kontschieder
This paper addresses common issues in 3D Gaussian Splatting (3DGS) such as blur, imperfect camera poses, and color inconsistencies, aiming to improve its robustness for practical applications like reconstructions from handheld phone captures. The main contributions include:
1. **Motion Blur Modeling**: Motion blur is modeled as a Gaussian distribution over camera poses, allowing for both camera pose refinement and motion blur correction. This approach maintains the benefits of 3DGS in terms of training efficiency and rendering speed.
2. **Defocus Blur Compensation**: An offset correction mechanism is proposed to address defocus blur, implemented via another covariance to the Gaussians once projected to the 2D image plane.
3. **Color Inconsistency Handling**: An RGB decoder function with per-image parameters is introduced to address color inconsistencies caused by ambient light, shadows, or camera-specific issues like varying white balancing settings.
4. **Evaluation**: The proposed methods are experimentally validated on benchmark datasets including Scannet++ and Deblur-NeRF, showing state-of-the-art results and consistent improvements over relevant baselines.
The paper also discusses related work, provides a brief review of Gaussian splatting, and includes detailed experimental results and ablation studies to demonstrate the effectiveness of the proposed techniques.This paper addresses common issues in 3D Gaussian Splatting (3DGS) such as blur, imperfect camera poses, and color inconsistencies, aiming to improve its robustness for practical applications like reconstructions from handheld phone captures. The main contributions include:
1. **Motion Blur Modeling**: Motion blur is modeled as a Gaussian distribution over camera poses, allowing for both camera pose refinement and motion blur correction. This approach maintains the benefits of 3DGS in terms of training efficiency and rendering speed.
2. **Defocus Blur Compensation**: An offset correction mechanism is proposed to address defocus blur, implemented via another covariance to the Gaussians once projected to the 2D image plane.
3. **Color Inconsistency Handling**: An RGB decoder function with per-image parameters is introduced to address color inconsistencies caused by ambient light, shadows, or camera-specific issues like varying white balancing settings.
4. **Evaluation**: The proposed methods are experimentally validated on benchmark datasets including Scannet++ and Deblur-NeRF, showing state-of-the-art results and consistent improvements over relevant baselines.
The paper also discusses related work, provides a brief review of Gaussian splatting, and includes detailed experimental results and ablation studies to demonstrate the effectiveness of the proposed techniques.