InFusion: Inpainting 3D Gaussians via Learning Depth Completion from Diffusion Prior

InFusion: Inpainting 3D Gaussians via Learning Depth Completion from Diffusion Prior

2024-04-17 | Zhiheng Liu, Hao Ouyang, Qiuyu Wang, Ka Leong Cheng, Jie Xiao, Kai Zhu, Nan Xue, Yu Liu, Yujun Shen, and Yang Cao
The paper "InFusion: Inpainting 3D Gaussians via Learning Depth Completion from Diffusion Prior" addresses the challenge of editing 3D scenes using Gaussian models, particularly focusing on the inpainting task. The authors propose a method called InFusion, which leverages depth completion learned from diffusion models to guide the initialization of points in 3D Gaussians. This approach enhances the fidelity and efficiency of 3D Gaussian inpainting, outperforming existing methods by achieving better visual quality and significantly faster processing times (up to 20 times faster). InFusion is demonstrated to be effective in various complex scenarios, including user-specific texture editing and object insertion. The method is based on the idea that precise depth completion can significantly improve the placement of initial points, leading to more coherent and realistic 3D scene editing. The paper also includes a detailed description of the method, experimental setup, and evaluation results, showing that InFusion outperforms baseline methods in both quantitative and qualitative metrics.The paper "InFusion: Inpainting 3D Gaussians via Learning Depth Completion from Diffusion Prior" addresses the challenge of editing 3D scenes using Gaussian models, particularly focusing on the inpainting task. The authors propose a method called InFusion, which leverages depth completion learned from diffusion models to guide the initialization of points in 3D Gaussians. This approach enhances the fidelity and efficiency of 3D Gaussian inpainting, outperforming existing methods by achieving better visual quality and significantly faster processing times (up to 20 times faster). InFusion is demonstrated to be effective in various complex scenarios, including user-specific texture editing and object insertion. The method is based on the idea that precise depth completion can significantly improve the placement of initial points, leading to more coherent and realistic 3D scene editing. The paper also includes a detailed description of the method, experimental setup, and evaluation results, showing that InFusion outperforms baseline methods in both quantitative and qualitative metrics.
Reach us at info@study.space