SemGauss-SLAM: Dense Semantic Gaussian Splatting SLAM

SemGauss-SLAM: Dense Semantic Gaussian Splatting SLAM

29 May 2024 | Siting Zhu, Renjie Qin, Guangming Wang, Jiuming Liu, Hesheng Wang
SemGauss-SLAM is a dense semantic SLAM system that uses 3D Gaussian representation to enable accurate 3D semantic mapping, robust camera tracking, and high-quality rendering. The system incorporates semantic feature embedding into 3D Gaussian representation, which encodes semantic information within the spatial layout of the environment for precise semantic scene representation. A feature-level loss is introduced to provide higher-level guidance for 3D Gaussian optimization. Additionally, semantic-informed bundle adjustment is introduced to reduce cumulative drift in tracking and improve semantic reconstruction accuracy by leveraging multi-frame semantic associations for joint optimization of 3D Gaussian representation and camera poses. This design exploits the consistency of multi-view semantics for establishing constraints, which enables the reduction of cumulative drift in tracking and enhanced semantic mapping precision. SemGauss-SLAM demonstrates superior performance over existing radiance field-based SLAM methods in terms of mapping and tracking accuracy on Replica and ScanNet datasets, while also showing excellent capabilities in high-precision semantic segmentation and dense semantic mapping. The system achieves accurate semantic mapping, photo-realistic reconstruction, and robust tracking by integrating semantic feature embedding into 3D Gaussian for semantic modeling. Moreover, semantic-informed bundle adjustment is used to achieve low-drift tracking and accurate semantic mapping. The method is evaluated on two challenging datasets, Replica and ScanNet, demonstrating state-of-the-art performance compared with existing radiance field-based SLAM in mapping, tracking, semantic segmentation, and novel-view synthesis. The system outperforms baseline methods in all metrics, achieving higher accuracy in semantic segmentation and novel view synthesis. The method also shows superior performance in semantic segmentation and novel view synthesis compared to other semantic SLAM methods. The system leverages the explicit structure of 3D Gaussian for unbounded mapping and performs semantic-informed bundle adjustment utilizing multi-frame semantic constraints for conducting low-drift and high-quality dense semantic SLAM. The system achieves accurate semantic mapping, photo-realistic reconstruction, and robust tracking by integrating semantic feature embedding into 3D Gaussian for semantic modeling. The method is evaluated on two challenging datasets, Replica and ScanNet, demonstrating state-of-the-art performance compared with existing radiance field-based SLAM in mapping, tracking, semantic segmentation, and novel-view synthesis. The system outperforms baseline methods in all metrics, achieving higher accuracy in semantic segmentation and novel view synthesis.SemGauss-SLAM is a dense semantic SLAM system that uses 3D Gaussian representation to enable accurate 3D semantic mapping, robust camera tracking, and high-quality rendering. The system incorporates semantic feature embedding into 3D Gaussian representation, which encodes semantic information within the spatial layout of the environment for precise semantic scene representation. A feature-level loss is introduced to provide higher-level guidance for 3D Gaussian optimization. Additionally, semantic-informed bundle adjustment is introduced to reduce cumulative drift in tracking and improve semantic reconstruction accuracy by leveraging multi-frame semantic associations for joint optimization of 3D Gaussian representation and camera poses. This design exploits the consistency of multi-view semantics for establishing constraints, which enables the reduction of cumulative drift in tracking and enhanced semantic mapping precision. SemGauss-SLAM demonstrates superior performance over existing radiance field-based SLAM methods in terms of mapping and tracking accuracy on Replica and ScanNet datasets, while also showing excellent capabilities in high-precision semantic segmentation and dense semantic mapping. The system achieves accurate semantic mapping, photo-realistic reconstruction, and robust tracking by integrating semantic feature embedding into 3D Gaussian for semantic modeling. Moreover, semantic-informed bundle adjustment is used to achieve low-drift tracking and accurate semantic mapping. The method is evaluated on two challenging datasets, Replica and ScanNet, demonstrating state-of-the-art performance compared with existing radiance field-based SLAM in mapping, tracking, semantic segmentation, and novel-view synthesis. The system outperforms baseline methods in all metrics, achieving higher accuracy in semantic segmentation and novel view synthesis. The method also shows superior performance in semantic segmentation and novel view synthesis compared to other semantic SLAM methods. The system leverages the explicit structure of 3D Gaussian for unbounded mapping and performs semantic-informed bundle adjustment utilizing multi-frame semantic constraints for conducting low-drift and high-quality dense semantic SLAM. The system achieves accurate semantic mapping, photo-realistic reconstruction, and robust tracking by integrating semantic feature embedding into 3D Gaussian for semantic modeling. The method is evaluated on two challenging datasets, Replica and ScanNet, demonstrating state-of-the-art performance compared with existing radiance field-based SLAM in mapping, tracking, semantic segmentation, and novel-view synthesis. The system outperforms baseline methods in all metrics, achieving higher accuracy in semantic segmentation and novel view synthesis.
Reach us at info@study.space