The paper introduces LIV-GaussMap, an integrated LiDAR-inertial-visual (LIV) multimodal sensor fusion system designed for real-time 3D radiance field map rendering. This system leverages the complementary strengths of LiDAR and visual data to capture the geometric structures of large-scale 3D scenes and restore their visual surface information with high fidelity. The initialization of scene surface Gaussians and sensor poses is achieved using a LiDAR-inertial system with size-adaptive voxels. Visual-derived photometric gradients are then used to optimize the quality and density of the Gaussians.
The method is compatible with various types of LiDAR, including solid-state and mechanical LiDAR, supporting both repetitive and non-repetitive scanning modes. It demonstrates notable resilience and versatility in generating real-time photorealistic scenes, which can be useful for digital twins, virtual reality, real-time SLAM, and robotics.
The system's effectiveness is evaluated on diverse public and private datasets, including both indoor and outdoor environments. The results show that the proposed method outperforms existing techniques in terms of peak signal-to-noise ratio (PSNR) and structural accuracy, particularly in extrapolated perspectives. The software and hardware, along with self-collected datasets, are released on GitHub to benefit the community.The paper introduces LIV-GaussMap, an integrated LiDAR-inertial-visual (LIV) multimodal sensor fusion system designed for real-time 3D radiance field map rendering. This system leverages the complementary strengths of LiDAR and visual data to capture the geometric structures of large-scale 3D scenes and restore their visual surface information with high fidelity. The initialization of scene surface Gaussians and sensor poses is achieved using a LiDAR-inertial system with size-adaptive voxels. Visual-derived photometric gradients are then used to optimize the quality and density of the Gaussians.
The method is compatible with various types of LiDAR, including solid-state and mechanical LiDAR, supporting both repetitive and non-repetitive scanning modes. It demonstrates notable resilience and versatility in generating real-time photorealistic scenes, which can be useful for digital twins, virtual reality, real-time SLAM, and robotics.
The system's effectiveness is evaluated on diverse public and private datasets, including both indoor and outdoor environments. The results show that the proposed method outperforms existing techniques in terms of peak signal-to-noise ratio (PSNR) and structural accuracy, particularly in extrapolated perspectives. The software and hardware, along with self-collected datasets, are released on GitHub to benefit the community.