**AutoSplat: Constrained Gaussian Splatting for Autonomous Driving Scene Reconstruction** **Authors:** Mustafa Khan, Hamidreza Fazlali, Dhruv Sharma, Tongtong Cao, Dongfeng Bai, Yuan Ren, Bingbing Liu **Institution:** University of Toronto, Noah’s Ark Lab, Huawei Technologies **Abstract:** Realistic scene reconstruction and view synthesis are crucial for advancing autonomous driving systems by simulating safety-critical scenarios. While 3D Gaussian Splatting (3DGS) excels in real-time rendering and static scene reconstructions, it struggles with dynamic objects and complex backgrounds in driving scenarios. This paper introduces AutoSplat, a framework that employs 3D Gaussian splatting to achieve highly realistic reconstructions of autonomous driving scenes. By imposing geometric constraints on Gaussians representing the road and sky regions, AutoSplat enables multi-view consistent simulation of challenging scenarios, including lane changes. Leveraging 3D templates, the method introduces a reflected Gaussian consistency constraint to supervise both visible and unseen sides of foreground objects. Additionally, residual spherical harmonics are estimated for each foreground Gaussian to model dynamic appearances. Extensive experiments on the Pandaset and KITTI datasets demonstrate that AutoSplat outperforms state-of-the-art methods in scene reconstruction and novel view synthesis across diverse driving scenarios. **Keywords:** Scene reconstruction, Novel view synthesis, Autonomous driving, 3D Gaussian Splatting **Introduction:** View synthesis and scene reconstruction from captured images are fundamental challenges in computer graphics and computer vision, crucial for autonomous driving and robotics. Reconstructing detailed 3D scenes from sparse sensor data on moving vehicles is particularly challenging at high speeds. While Neural Radiance Fields (NeRFs) have transformed view synthesis and reconstruction, they face challenges such as slow training and rendering speed, high memory usage, and imprecise geometry estimation, especially with sparse viewpoints. In contrast, 3D Gaussian Splatting (3DGS) explicitly represents the scene using anisotropic 3D Gaussians, enabling faster training and high-quality novel view synthesis. However, 3DGS struggles with dynamic objects and complex backgrounds in autonomous driving scenarios. **Method:** AutoSplat decomposes the background and geometrically constrains its road and sky regions to enable multi-view consistent rasterization. It leverages 3D templates for initializing foreground Gaussians, paired with a reflected Gaussian consistency constraint to reconstruct unseen parts from symmetrically visible views. The method captures the dynamic visual characteristics of foreground objects through the estimation of temporally-dependent, residual spherical harmonics. Comprehensive evaluation on the Pandaset and KITTI datasets demonstrates the effectiveness of AutoSplat in scene reconstruction and novel view synthesis. **Experiments:** AutoSplat is evaluated on the Pandaset and KITTI datasets, showing superior performance in scene reconstruction and novel view synthesis compared to state-of-the-art methods. Ablation studies validate the**AutoSplat: Constrained Gaussian Splatting for Autonomous Driving Scene Reconstruction** **Authors:** Mustafa Khan, Hamidreza Fazlali, Dhruv Sharma, Tongtong Cao, Dongfeng Bai, Yuan Ren, Bingbing Liu **Institution:** University of Toronto, Noah’s Ark Lab, Huawei Technologies **Abstract:** Realistic scene reconstruction and view synthesis are crucial for advancing autonomous driving systems by simulating safety-critical scenarios. While 3D Gaussian Splatting (3DGS) excels in real-time rendering and static scene reconstructions, it struggles with dynamic objects and complex backgrounds in driving scenarios. This paper introduces AutoSplat, a framework that employs 3D Gaussian splatting to achieve highly realistic reconstructions of autonomous driving scenes. By imposing geometric constraints on Gaussians representing the road and sky regions, AutoSplat enables multi-view consistent simulation of challenging scenarios, including lane changes. Leveraging 3D templates, the method introduces a reflected Gaussian consistency constraint to supervise both visible and unseen sides of foreground objects. Additionally, residual spherical harmonics are estimated for each foreground Gaussian to model dynamic appearances. Extensive experiments on the Pandaset and KITTI datasets demonstrate that AutoSplat outperforms state-of-the-art methods in scene reconstruction and novel view synthesis across diverse driving scenarios. **Keywords:** Scene reconstruction, Novel view synthesis, Autonomous driving, 3D Gaussian Splatting **Introduction:** View synthesis and scene reconstruction from captured images are fundamental challenges in computer graphics and computer vision, crucial for autonomous driving and robotics. Reconstructing detailed 3D scenes from sparse sensor data on moving vehicles is particularly challenging at high speeds. While Neural Radiance Fields (NeRFs) have transformed view synthesis and reconstruction, they face challenges such as slow training and rendering speed, high memory usage, and imprecise geometry estimation, especially with sparse viewpoints. In contrast, 3D Gaussian Splatting (3DGS) explicitly represents the scene using anisotropic 3D Gaussians, enabling faster training and high-quality novel view synthesis. However, 3DGS struggles with dynamic objects and complex backgrounds in autonomous driving scenarios. **Method:** AutoSplat decomposes the background and geometrically constrains its road and sky regions to enable multi-view consistent rasterization. It leverages 3D templates for initializing foreground Gaussians, paired with a reflected Gaussian consistency constraint to reconstruct unseen parts from symmetrically visible views. The method captures the dynamic visual characteristics of foreground objects through the estimation of temporally-dependent, residual spherical harmonics. Comprehensive evaluation on the Pandaset and KITTI datasets demonstrates the effectiveness of AutoSplat in scene reconstruction and novel view synthesis. **Experiments:** AutoSplat is evaluated on the Pandaset and KITTI datasets, showing superior performance in scene reconstruction and novel view synthesis compared to state-of-the-art methods. Ablation studies validate the