LetsGo: Large-Scale Garage Modeling and Rendering via LiDAR-Assisted Gaussian Primitives

LetsGo: Large-Scale Garage Modeling and Rendering via LiDAR-Assisted Gaussian Primitives

21 May 2024 | Jiadi Cui, Junming Cao, Fuqiang Zhao, Zhipeng He, Yifan Chen, Yuhui Zhong, Lan Xu, Yujiao Shi, Yingliang Zhang, Jingyi Yu
This paper introduces LetsGo, an explicit and efficient end-to-end framework for high-fidelity rendering of large-scale garages. The framework leverages LiDAR-assisted Gaussian primitives and a newly developed dataset, GarageWorld, which contains eight large-scale garage scenes with diverse geometric structures. The framework addresses the challenges of modeling and rendering garages, which are often characterized by monotonous colors, repetitive patterns, reflective surfaces, and transparent vehicle glass. The framework uses a handheld Polar scanner equipped with IMU, LiDAR, and a fisheye camera to capture RGBD data of expansive parking environments. The Polar device enables accurate data acquisition and facilitates the creation of the GarageWorld dataset, which will be made publicly available for further research. The framework introduces a novel depth regularizer that effectively eliminates floating artifacts in rendered images. Additionally, it proposes a multi-resolution 3D Gaussian representation designed for Level-of-Detail (LOD) rendering. This representation includes adapted scaling factors for individual levels and a random-resolution-level training scheme to optimize the Gaussians across different resolutions. This representation enables efficient rendering of large-scale garage scenes on lightweight devices via a web-based renderer. The framework demonstrates superior rendering quality and resource efficiency on the GarageWorld dataset, as well as on ScanNet++ and KITTI-360 datasets. The framework enables various applications, including autonomous driving, localization, navigation, and visual effects. The framework also introduces a multi-resolution Gaussian representation that allows for efficient rendering of large-scale scenes on lightweight devices. The framework's web-based renderer supports LOD rendering across various consumer-level devices, including laptops and tablets. The framework's source codes, including training code, high-performance PC viewer, and lightweight web viewer, will be released to facilitate reproducible research.This paper introduces LetsGo, an explicit and efficient end-to-end framework for high-fidelity rendering of large-scale garages. The framework leverages LiDAR-assisted Gaussian primitives and a newly developed dataset, GarageWorld, which contains eight large-scale garage scenes with diverse geometric structures. The framework addresses the challenges of modeling and rendering garages, which are often characterized by monotonous colors, repetitive patterns, reflective surfaces, and transparent vehicle glass. The framework uses a handheld Polar scanner equipped with IMU, LiDAR, and a fisheye camera to capture RGBD data of expansive parking environments. The Polar device enables accurate data acquisition and facilitates the creation of the GarageWorld dataset, which will be made publicly available for further research. The framework introduces a novel depth regularizer that effectively eliminates floating artifacts in rendered images. Additionally, it proposes a multi-resolution 3D Gaussian representation designed for Level-of-Detail (LOD) rendering. This representation includes adapted scaling factors for individual levels and a random-resolution-level training scheme to optimize the Gaussians across different resolutions. This representation enables efficient rendering of large-scale garage scenes on lightweight devices via a web-based renderer. The framework demonstrates superior rendering quality and resource efficiency on the GarageWorld dataset, as well as on ScanNet++ and KITTI-360 datasets. The framework enables various applications, including autonomous driving, localization, navigation, and visual effects. The framework also introduces a multi-resolution Gaussian representation that allows for efficient rendering of large-scale scenes on lightweight devices. The framework's web-based renderer supports LOD rendering across various consumer-level devices, including laptops and tablets. The framework's source codes, including training code, high-performance PC viewer, and lightweight web viewer, will be released to facilitate reproducible research.
Reach us at info@study.space