11 Mar 2024 | Yifu Tao1, Yash Bhalgat2, Lanke Frank Tarimo Fu1, Matias Mattamala1, Nived Chebrolu1, and Maurice Fallon1
The paper presents a large-scale 3D reconstruction system that integrates lidar and vision data to generate high-quality, geometrically accurate, and photo-realistic textures. The system leverages neural radiance fields (NeRF) to optimize a continuous 3D representation, incorporating lidar data for geometric constraints on depth and surface normals. The authors exploit the trajectory from a real-time lidar SLAM system to reduce computation time and provide metric scale, crucial for lidar depth loss. Submapping is used to scale the system to large environments captured over long trajectories. The system is demonstrated using data from a multi-camera, lidar sensor suite on a legged robot, a handheld device, and an aerial robot. The main contributions include a dense textured 3D reconstruction system, integration with a lidar SLAM system, sub-mapping for large-scale environments, and evaluation on real-world datasets. The method improves reconstruction quality in textureless areas and provides accurate geometry comparable to lidar, with photorealistic novel view synthesis.The paper presents a large-scale 3D reconstruction system that integrates lidar and vision data to generate high-quality, geometrically accurate, and photo-realistic textures. The system leverages neural radiance fields (NeRF) to optimize a continuous 3D representation, incorporating lidar data for geometric constraints on depth and surface normals. The authors exploit the trajectory from a real-time lidar SLAM system to reduce computation time and provide metric scale, crucial for lidar depth loss. Submapping is used to scale the system to large environments captured over long trajectories. The system is demonstrated using data from a multi-camera, lidar sensor suite on a legged robot, a handheld device, and an aerial robot. The main contributions include a dense textured 3D reconstruction system, integration with a lidar SLAM system, sub-mapping for large-scale environments, and evaluation on real-world datasets. The method improves reconstruction quality in textureless areas and provides accurate geometry comparable to lidar, with photorealistic novel view synthesis.