Splat-SLAM: Globally Optimized RGB-only SLAM with 3D Gaussians

Splat-SLAM: Globally Optimized RGB-only SLAM with 3D Gaussians

26 May 2024 | Erik Sandström, Keisuke Tateno, Michael Oechsle, Michael Niemeyer, Luc Van Gool, Martin R. Oswald, Federico Tombari
Splat-SLAM is a globally optimized RGB-only SLAM system that uses a dense 3D Gaussian map for mapping and globally optimized frame-to-frame tracking. The system improves reconstruction accuracy by incorporating monocular depth estimation and dynamically deforming the 3D Gaussian map to adapt to keyframe pose and depth updates. This approach enables accurate surface reconstruction and high-quality rendering while maintaining small map sizes and fast runtimes. The method outperforms existing RGB-only SLAM approaches on the Replica, TUM-RGBD, and ScanNet datasets, achieving superior or comparable performance in terms of mapping and rendering accuracy. The system uses a combination of frame-to-frame tracking with recurrent dense optical flow and the fidelity of 3D Gaussians as the map representation. The 3D Gaussian map is optimized through a re-rendering loss and is adjusted for global pose and depth updates before each mapping phase. The system also incorporates a proxy depth map that combines multi-view depth estimation with learned monocular depth to improve rendering and reconstruction quality. The method is evaluated on multiple datasets and shows significant improvements in rendering accuracy, reconstruction quality, and runtime efficiency compared to other dense SLAM approaches. The source code is available at https://github.com/eriksandstroem/Splat-SLAM.Splat-SLAM is a globally optimized RGB-only SLAM system that uses a dense 3D Gaussian map for mapping and globally optimized frame-to-frame tracking. The system improves reconstruction accuracy by incorporating monocular depth estimation and dynamically deforming the 3D Gaussian map to adapt to keyframe pose and depth updates. This approach enables accurate surface reconstruction and high-quality rendering while maintaining small map sizes and fast runtimes. The method outperforms existing RGB-only SLAM approaches on the Replica, TUM-RGBD, and ScanNet datasets, achieving superior or comparable performance in terms of mapping and rendering accuracy. The system uses a combination of frame-to-frame tracking with recurrent dense optical flow and the fidelity of 3D Gaussians as the map representation. The 3D Gaussian map is optimized through a re-rendering loss and is adjusted for global pose and depth updates before each mapping phase. The system also incorporates a proxy depth map that combines multi-view depth estimation with learned monocular depth to improve rendering and reconstruction quality. The method is evaluated on multiple datasets and shows significant improvements in rendering accuracy, reconstruction quality, and runtime efficiency compared to other dense SLAM approaches. The source code is available at https://github.com/eriksandstroem/Splat-SLAM.
Reach us at info@study.space
Understanding Splat-SLAM%3A Globally Optimized RGB-only SLAM with 3D Gaussians