LetsGo: Large-Scale Garage Modeling and Rendering via LiDAR-Assisted Gaussian Primitives

LetsGo: Large-Scale Garage Modeling and Rendering via LiDAR-Assisted Gaussian Primitives

21 May 2024 | Jiadi Cui, Junming Cao, Fuqiang Zhao, Zhipeng He, Yifan Chen, Yuhui Zhong, Lan Xu, Yujiao Shi, Yingliang Zhang, Jingyi Yu
The paper introduces LetsGo, an end-to-end framework for high-fidelity rendering of large-scale garages using LiDAR-assisted Gaussian primitives. The authors designed a handheld Polar scanner equipped with IMU, LiDAR, and a fisheye camera to capture RGBD data of expansive parking environments. They scanned eight garages, creating the GarageWorld dataset, which is the first of its kind for large-scale garages. The dataset includes various geometric structures such as underground garages, multi-floor indoor garages, and outdoor parking lots. The key innovation of LetsGo is the integration of calibrated LiDAR points into 3D Gaussian splatting algorithms. This approach enhances the realism and efficiency of 3D scene modeling and rendering. The authors introduced a depth regularizer that uses depth priors from the Polar device to reduce floating artifacts in rendered images. Additionally, they proposed a multi-resolution 3D Gaussian representation for Level-of-Detail (LOD) rendering, which dynamically adjusts based on the camera's position and orientation. The paper also discusses the challenges of capturing and rendering garage environments, such as low lighting, textureless surfaces, and reflective materials. Traditional methods like Structure from Motion (SfM) and Multi-View Stereo (MVS) often struggle in these environments due to poor feature correspondences. The authors compared their method with state-of-the-art techniques on various datasets, demonstrating superior rendering quality and efficiency. The GarageWorld dataset and LetsGo framework enable applications such as autonomous vehicle localization, navigation, and parking, as well as visual effects production. The authors plan to release their source codes and datasets to facilitate further research.The paper introduces LetsGo, an end-to-end framework for high-fidelity rendering of large-scale garages using LiDAR-assisted Gaussian primitives. The authors designed a handheld Polar scanner equipped with IMU, LiDAR, and a fisheye camera to capture RGBD data of expansive parking environments. They scanned eight garages, creating the GarageWorld dataset, which is the first of its kind for large-scale garages. The dataset includes various geometric structures such as underground garages, multi-floor indoor garages, and outdoor parking lots. The key innovation of LetsGo is the integration of calibrated LiDAR points into 3D Gaussian splatting algorithms. This approach enhances the realism and efficiency of 3D scene modeling and rendering. The authors introduced a depth regularizer that uses depth priors from the Polar device to reduce floating artifacts in rendered images. Additionally, they proposed a multi-resolution 3D Gaussian representation for Level-of-Detail (LOD) rendering, which dynamically adjusts based on the camera's position and orientation. The paper also discusses the challenges of capturing and rendering garage environments, such as low lighting, textureless surfaces, and reflective materials. Traditional methods like Structure from Motion (SfM) and Multi-View Stereo (MVS) often struggle in these environments due to poor feature correspondences. The authors compared their method with state-of-the-art techniques on various datasets, demonstrating superior rendering quality and efficiency. The GarageWorld dataset and LetsGo framework enable applications such as autonomous vehicle localization, navigation, and parking, as well as visual effects production. The authors plan to release their source codes and datasets to facilitate further research.
Reach us at info@study.space