Learning 3D Geometry and Feature Consistent Gaussian Splatting for Object Removal

Learning 3D Geometry and Feature Consistent Gaussian Splatting for Object Removal

21 Apr 2024 | Yuxin Wang, Qianyi Wu, Guofeng Zhang, Dan Xu
This paper addresses the challenge of object removal in 3D scenes using 3D Gaussian Splatting (3DGS). The main challenges include preserving geometric consistency and maintaining texture coherence, which are exacerbated by the discrete nature of Gaussian primitives. The authors propose a robust framework, GScream, that enhances information exchange between visible and invisible areas to improve content restoration in terms of geometry and texture. Key contributions include: 1. **Geometric Consistency**: The method incorporates monocular depth estimation from multi-view images to guide the placement of Gaussian primitives, ensuring precise alignment and improved geometric accuracy. 2. **Texture Coherence**: A novel feature propagation mechanism, leveraging cross-attention, is introduced to enhance texture coherence across different viewing angles. This mechanism facilitates the blending of visible and in-painted regions, ensuring seamless appearance transitions. 3. **Efficiency**: The use of 3DGS, combined with a lightweight Gaussian Splatting architecture (Scaffold-GS), significantly reduces computational costs, making the method suitable for practical applications. Experiments on the SPIn-NeRF and IBRNet datasets demonstrate that GScream outperforms existing methods in terms of both quality and efficiency, achieving superior results in novel view synthesis and faster training and rendering speeds.This paper addresses the challenge of object removal in 3D scenes using 3D Gaussian Splatting (3DGS). The main challenges include preserving geometric consistency and maintaining texture coherence, which are exacerbated by the discrete nature of Gaussian primitives. The authors propose a robust framework, GScream, that enhances information exchange between visible and invisible areas to improve content restoration in terms of geometry and texture. Key contributions include: 1. **Geometric Consistency**: The method incorporates monocular depth estimation from multi-view images to guide the placement of Gaussian primitives, ensuring precise alignment and improved geometric accuracy. 2. **Texture Coherence**: A novel feature propagation mechanism, leveraging cross-attention, is introduced to enhance texture coherence across different viewing angles. This mechanism facilitates the blending of visible and in-painted regions, ensuring seamless appearance transitions. 3. **Efficiency**: The use of 3DGS, combined with a lightweight Gaussian Splatting architecture (Scaffold-GS), significantly reduces computational costs, making the method suitable for practical applications. Experiments on the SPIn-NeRF and IBRNet datasets demonstrate that GScream outperforms existing methods in terms of both quality and efficiency, achieving superior results in novel view synthesis and faster training and rendering speeds.
Reach us at info@study.space
Understanding GScream%3A Learning 3D Geometry and Feature Consistent Gaussian Splatting for Object Removal