LIV-GaussMap: LiDAR-Inertial-Visual Fusion for Real-time 3D Radiance Field Map Rendering

LIV-GaussMap: LiDAR-Inertial-Visual Fusion for Real-time 3D Radiance Field Map Rendering

Accepted April 2024 | Sheng Hong1,*., Junjie He2,*., Xinhu Zheng2., Chunran Zheng3., Shaojie Shen1†
The paper introduces LIV-GaussMap, an integrated LiDAR-inertial-visual (LIV) multimodal sensor fusion system designed for real-time 3D radiance field map rendering. This system leverages the complementary strengths of LiDAR and visual data to capture the geometric structures of large-scale 3D scenes and restore their visual surface information with high fidelity. The initialization of scene surface Gaussians and sensor poses is achieved using a LiDAR-inertial system with size-adaptive voxels. Visual-derived photometric gradients are then used to optimize the quality and density of the Gaussians. The method is compatible with various types of LiDAR, including solid-state and mechanical LiDAR, supporting both repetitive and non-repetitive scanning modes. It demonstrates notable resilience and versatility in generating real-time photorealistic scenes, which can be useful for digital twins, virtual reality, real-time SLAM, and robotics. The system's effectiveness is evaluated on diverse public and private datasets, including both indoor and outdoor environments. The results show that the proposed method outperforms existing techniques in terms of peak signal-to-noise ratio (PSNR) and structural accuracy, particularly in extrapolated perspectives. The software and hardware, along with self-collected datasets, are released on GitHub to benefit the community.The paper introduces LIV-GaussMap, an integrated LiDAR-inertial-visual (LIV) multimodal sensor fusion system designed for real-time 3D radiance field map rendering. This system leverages the complementary strengths of LiDAR and visual data to capture the geometric structures of large-scale 3D scenes and restore their visual surface information with high fidelity. The initialization of scene surface Gaussians and sensor poses is achieved using a LiDAR-inertial system with size-adaptive voxels. Visual-derived photometric gradients are then used to optimize the quality and density of the Gaussians. The method is compatible with various types of LiDAR, including solid-state and mechanical LiDAR, supporting both repetitive and non-repetitive scanning modes. It demonstrates notable resilience and versatility in generating real-time photorealistic scenes, which can be useful for digital twins, virtual reality, real-time SLAM, and robotics. The system's effectiveness is evaluated on diverse public and private datasets, including both indoor and outdoor environments. The results show that the proposed method outperforms existing techniques in terms of peak signal-to-noise ratio (PSNR) and structural accuracy, particularly in extrapolated perspectives. The software and hardware, along with self-collected datasets, are released on GitHub to benefit the community.
Reach us at info@study.space
[slides] LIV-GaussMap%3A LiDAR-Inertial-Visual Fusion for Real-Time 3D Radiance Field Map Rendering | StudySpace