Gaussian in the Wild: 3D Gaussian Splatting for Unconstrained Image Collections

Gaussian in the Wild: 3D Gaussian Splatting for Unconstrained Image Collections

14 Jul 2024 | Dongbin Zhang*, Chuming Wang*, Weitao Wang, Peihao Li, Minghan Qin, and Haoqian Wang†
Gaussian in the Wild (GS-W) is a method for reconstructing 3D scenes from unconstrained image collections. It uses 3D Gaussian points to represent the scene and introduces separated intrinsic and dynamic appearance features for each point, capturing both static and dynamic variations in appearance. GS-W also employs an adaptive sampling strategy to focus on local and detailed information and uses a 2D visibility map to reduce the impact of transient occluders. Experimental results show that GS-W outperforms NeRF-based methods in terms of reconstruction quality and rendering speed, achieving state-of-the-art performance. The method is efficient and can render novel views with appearance tuning, making it suitable for applications like virtual reality and autonomous driving. GS-W is implemented using PyTorch and trained on a single GPU, achieving a rendering speed of 200 FPS. The method is effective in handling complex lighting variations, specular reflections, and accurately reconstructing textures in frequently occluded scenes. GS-W also allows for appearance tuning by adjusting the impact of dynamic appearance features on intrinsic appearance. The method is validated through extensive experiments on three PhotoTourism scenes, demonstrating its superiority in capturing fine details and maintaining multi-view consistency. GS-W is a promising approach for novel view synthesis from unconstrained image collections.Gaussian in the Wild (GS-W) is a method for reconstructing 3D scenes from unconstrained image collections. It uses 3D Gaussian points to represent the scene and introduces separated intrinsic and dynamic appearance features for each point, capturing both static and dynamic variations in appearance. GS-W also employs an adaptive sampling strategy to focus on local and detailed information and uses a 2D visibility map to reduce the impact of transient occluders. Experimental results show that GS-W outperforms NeRF-based methods in terms of reconstruction quality and rendering speed, achieving state-of-the-art performance. The method is efficient and can render novel views with appearance tuning, making it suitable for applications like virtual reality and autonomous driving. GS-W is implemented using PyTorch and trained on a single GPU, achieving a rendering speed of 200 FPS. The method is effective in handling complex lighting variations, specular reflections, and accurately reconstructing textures in frequently occluded scenes. GS-W also allows for appearance tuning by adjusting the impact of dynamic appearance features on intrinsic appearance. The method is validated through extensive experiments on three PhotoTourism scenes, demonstrating its superiority in capturing fine details and maintaining multi-view consistency. GS-W is a promising approach for novel view synthesis from unconstrained image collections.
Reach us at info@study.space
Understanding Gaussian in the Wild%3A 3D Gaussian Splatting for Unconstrained Image Collections