April 2024 | XIAOYANG LYU and YANG-TIAN SUN, The University of Hong Kong, Hong Kong; YI-HUA HUANG and XIUZHE WU, The University of Hong Kong, Hong Kong; ZIYI YANG, Zhejiang University, China; YILUN CHEN and JIANGMIAO PANG, Shanghai AI Lab, China; XIAOJUAN QI, The University of Hong Kong, Hong Kong
This paper introduces 3DGSR, an implicit surface reconstruction method that integrates 3D Gaussian Splatting (3DGS) with a neural implicit signed distance field (SDF). The method enables accurate 3D surface reconstruction with rich details while maintaining the efficiency and high-quality rendering of 3DGS. The key idea is to incorporate an implicit SDF within 3D Gaussians to align and jointly optimize them. A differentiable SDF-to-opacity transformation function is introduced to convert SDF values into corresponding Gaussians' opacities, connecting SDF and Gaussians for unified optimization and enforcing surface constraints. During learning, optimizing the Gaussians provides supervisory signals for SDF learning, enabling the reconstruction of intricate details. However, this only provides sparse supervisory signals to the SDF at locations occupied by Gaussians, which is insufficient for learning a continuous SDF. To address this limitation, volumetric rendering is incorporated, aligning the rendered geometric attributes (depth, normal) with those derived from 3D Gaussians. This consistency regularization introduces supervisory signals to locations not covered by discrete 3D Gaussians, effectively eliminating redundant surfaces outside the Gaussian sampling range. The method achieves high-quality 3D surface reconstruction while preserving the efficiency and rendering quality of 3DGS. Experimental results show that 3DGSR outperforms state-of-the-art methods in rendering quality and facilitates the reconstruction of intricate structures with significantly shorter learning time. The code is available at https://github.com/CVMI-Lab/3DGSR.This paper introduces 3DGSR, an implicit surface reconstruction method that integrates 3D Gaussian Splatting (3DGS) with a neural implicit signed distance field (SDF). The method enables accurate 3D surface reconstruction with rich details while maintaining the efficiency and high-quality rendering of 3DGS. The key idea is to incorporate an implicit SDF within 3D Gaussians to align and jointly optimize them. A differentiable SDF-to-opacity transformation function is introduced to convert SDF values into corresponding Gaussians' opacities, connecting SDF and Gaussians for unified optimization and enforcing surface constraints. During learning, optimizing the Gaussians provides supervisory signals for SDF learning, enabling the reconstruction of intricate details. However, this only provides sparse supervisory signals to the SDF at locations occupied by Gaussians, which is insufficient for learning a continuous SDF. To address this limitation, volumetric rendering is incorporated, aligning the rendered geometric attributes (depth, normal) with those derived from 3D Gaussians. This consistency regularization introduces supervisory signals to locations not covered by discrete 3D Gaussians, effectively eliminating redundant surfaces outside the Gaussian sampling range. The method achieves high-quality 3D surface reconstruction while preserving the efficiency and rendering quality of 3DGS. Experimental results show that 3DGSR outperforms state-of-the-art methods in rendering quality and facilitates the reconstruction of intricate structures with significantly shorter learning time. The code is available at https://github.com/CVMI-Lab/3DGSR.