9 Jun 2024 | Hanlin Chen, Fangyin Wei, Chen Li, Tianxin Huang, Yunsong Wang, Gim Hee Lee
VCR-GauS is a method for view-consistent depth-normal regularization in Gaussian surface reconstruction. The paper introduces a novel D-Normal regularizer that couples normal with other geometric parameters, enabling full updates of the geometric parameters from normal regularization. It also proposes a confidence term to mitigate inconsistencies of normal predictions across multiple views. Additionally, a densification and splitting strategy is introduced to regularize the size and distribution of 3D Gaussians for more accurate surface modeling. The method achieves new state-of-the-art surface reconstruction results and rendering quality comparable with prior work. Experiments show that the approach outperforms Gaussian-based baselines in terms of reconstruction quality and maintains competitive appearance quality at faster training speed and 100+ FPS rendering. The method is evaluated on benchmarking datasets including Tanks and Temples, Replica, MipNeRF360, and DTU, demonstrating superior performance in surface reconstruction and novel view synthesis. The method addresses two main limitations of previous Gaussian-based reconstruction: 1) supervision of normal rendered from 3D Gaussians only updates rotation parameters, neglecting other geometric parameters; 2) inconsistency of predicted normal maps across multiple views may lead to severe reconstruction artifacts. The proposed method introduces a depth-normal formulation where the normal is derived from the gradient of rendered depth instead of directly blended from 3D Gaussians. It also introduces a confidence term to weigh the D-Normal regularizer to mitigate inconsistencies of normal predictions across multiple views. The method also introduces a densification and splitting strategy to alleviate depth error towards more accurate surface modeling. The method is evaluated on various datasets and shows superior performance in surface reconstruction and novel view synthesis.VCR-GauS is a method for view-consistent depth-normal regularization in Gaussian surface reconstruction. The paper introduces a novel D-Normal regularizer that couples normal with other geometric parameters, enabling full updates of the geometric parameters from normal regularization. It also proposes a confidence term to mitigate inconsistencies of normal predictions across multiple views. Additionally, a densification and splitting strategy is introduced to regularize the size and distribution of 3D Gaussians for more accurate surface modeling. The method achieves new state-of-the-art surface reconstruction results and rendering quality comparable with prior work. Experiments show that the approach outperforms Gaussian-based baselines in terms of reconstruction quality and maintains competitive appearance quality at faster training speed and 100+ FPS rendering. The method is evaluated on benchmarking datasets including Tanks and Temples, Replica, MipNeRF360, and DTU, demonstrating superior performance in surface reconstruction and novel view synthesis. The method addresses two main limitations of previous Gaussian-based reconstruction: 1) supervision of normal rendered from 3D Gaussians only updates rotation parameters, neglecting other geometric parameters; 2) inconsistency of predicted normal maps across multiple views may lead to severe reconstruction artifacts. The proposed method introduces a depth-normal formulation where the normal is derived from the gradient of rendered depth instead of directly blended from 3D Gaussians. It also introduces a confidence term to weigh the D-Normal regularizer to mitigate inconsistencies of normal predictions across multiple views. The method also introduces a densification and splitting strategy to alleviate depth error towards more accurate surface modeling. The method is evaluated on various datasets and shows superior performance in surface reconstruction and novel view synthesis.