360-GS: Layout-guided Panoramic Gaussian Splatting For Indoor Roaming

360-GS: Layout-guided Panoramic Gaussian Splatting For Indoor Roaming

August 2024 | Jiayang Bai, Letian Huang, Jie Guo, Wen Gong, Yuanqi Li, Yanwen Guo
360-GS: Layout-guided Panoramic Gaussian Splatting for Indoor Roaming This paper proposes 360-GS, a novel layout-guided panoramic Gaussian splatting method for indoor roaming. The method addresses the challenges of applying 3D Gaussian Splatting (3D-GS) to panoramic inputs, which suffer from spatial distortion and under-constrained geometry. 360-GS introduces a new approach to splat 3D Gaussians onto the spherical surface of panoramic images by projecting them onto the tangent plane of the unit sphere and then mapping them to the spherical surface. This allows the representation of the projection using Gaussians. The method also incorporates room layout priors to guide the optimization of 360-GS, which are simple to obtain and contain strong structural information about the indoor scene. The experimental results demonstrate that 360-GS allows panoramic rendering and outperforms state-of-the-art methods with fewer artifacts in novel view synthesis, thus providing immersive roaming in indoor scenarios. The method is designed for sparse panoramic inputs and achieves real-time performance. The key components of 360-GS include 360° Gaussian splatting and the incorporation of room layout priors. The 360° Gaussian splatting algorithm decomposes the splatting into two steps: projecting 3D Gaussians onto the tangent plane and then mapping them to the spherical surface. The decomposition avoids the complicated representation of projections while maintaining real-time performance. The method also introduces a layout-guided regularization to reduce floaters caused by under-constrained regions. The experiments conducted on real-world datasets have demonstrated the superiority and effectiveness of our method. The main contributions of our paper are: (1) We propose 360-GS, a layout-guided 3D Gaussian splatting pipeline designed for sparse panoramic images, which allows real-time panoramic rendering using our novel 360° Gaussian splatting algorithm. (2) We derive a high-quality point cloud generation method for the initialization of 3D Gaussians from room layout priors to improve the performance of few-shot novel view synthesis. (3) We introduce a layout-guided regularization on 3D Gaussians to reduce floaters caused by under-constrained regions.360-GS: Layout-guided Panoramic Gaussian Splatting for Indoor Roaming This paper proposes 360-GS, a novel layout-guided panoramic Gaussian splatting method for indoor roaming. The method addresses the challenges of applying 3D Gaussian Splatting (3D-GS) to panoramic inputs, which suffer from spatial distortion and under-constrained geometry. 360-GS introduces a new approach to splat 3D Gaussians onto the spherical surface of panoramic images by projecting them onto the tangent plane of the unit sphere and then mapping them to the spherical surface. This allows the representation of the projection using Gaussians. The method also incorporates room layout priors to guide the optimization of 360-GS, which are simple to obtain and contain strong structural information about the indoor scene. The experimental results demonstrate that 360-GS allows panoramic rendering and outperforms state-of-the-art methods with fewer artifacts in novel view synthesis, thus providing immersive roaming in indoor scenarios. The method is designed for sparse panoramic inputs and achieves real-time performance. The key components of 360-GS include 360° Gaussian splatting and the incorporation of room layout priors. The 360° Gaussian splatting algorithm decomposes the splatting into two steps: projecting 3D Gaussians onto the tangent plane and then mapping them to the spherical surface. The decomposition avoids the complicated representation of projections while maintaining real-time performance. The method also introduces a layout-guided regularization to reduce floaters caused by under-constrained regions. The experiments conducted on real-world datasets have demonstrated the superiority and effectiveness of our method. The main contributions of our paper are: (1) We propose 360-GS, a layout-guided 3D Gaussian splatting pipeline designed for sparse panoramic images, which allows real-time panoramic rendering using our novel 360° Gaussian splatting algorithm. (2) We derive a high-quality point cloud generation method for the initialization of 3D Gaussians from room layout priors to improve the performance of few-shot novel view synthesis. (3) We introduce a layout-guided regularization on 3D Gaussians to reduce floaters caused by under-constrained regions.
Reach us at info@study.space
[slides] 360-GS%3A Layout-guided Panoramic Gaussian Splatting For Indoor Roaming | StudySpace