LaRa: Efficient Large-Baseline Radiance Fields

LaRa: Efficient Large-Baseline Radiance Fields

15 Jul 2024 | Anpei Chen, Haofei Xu, Stefano Esposito, Siyu Tang, Andreas Geiger
LaRa is a feed-forward 2D Gaussian Splatting model that reconstructs radiance fields from large-baseline views, a single image, or a text prompt. The method introduces a Gaussian volume as the 3D representation, where each voxel contains a set of learnable Gaussian primitives. The model combines an image encoder and Group Attention Layers for efficient feed-forward reconstruction. Experimental results show that the model, trained for two days on four GPUs, achieves high fidelity in reconstructing 360° radiance fields and robustness to zero-shot and out-of-domain testing. The model uses a coarse-to-fine decoding process to efficiently share information across neighboring groups and outputs 2D Gaussian parameters. LaRa's approach unifies local and global reasoning in transformer layers, resulting in improved quality and faster convergence. The model is trained on large-scale image collections and achieves high-quality reconstruction results using only 4 A100-40G GPUs within a span of 2 days. LaRa's method is efficient and robust, providing photorealistic, 360° novel view synthesis results using only four input images. The model also allows high-quality mesh reconstruction using off-the-shelf depth-map fusion algorithms. LaRa's approach is designed to handle unstructured views and is flexible to accommodate various numbers of views. The model's performance is evaluated on multiple datasets, including the Objaverse dataset, Google Scanned Objects dataset, and the Co3D test set. LaRa outperforms previous methods in terms of rendering quality, geometry accuracy, and generalization to real-world data. The model's effectiveness is demonstrated through quantitative results and visual comparisons with other methods. LaRa's approach is efficient and robust, providing high-quality 3D reconstructions with minimal training resources. The model's performance is further validated through ablation studies and comparisons with other methods. LaRa's method is a significant advancement in the field of radiance field reconstruction, offering a novel approach to large-baseline 3D reconstruction with high efficiency and accuracy.LaRa is a feed-forward 2D Gaussian Splatting model that reconstructs radiance fields from large-baseline views, a single image, or a text prompt. The method introduces a Gaussian volume as the 3D representation, where each voxel contains a set of learnable Gaussian primitives. The model combines an image encoder and Group Attention Layers for efficient feed-forward reconstruction. Experimental results show that the model, trained for two days on four GPUs, achieves high fidelity in reconstructing 360° radiance fields and robustness to zero-shot and out-of-domain testing. The model uses a coarse-to-fine decoding process to efficiently share information across neighboring groups and outputs 2D Gaussian parameters. LaRa's approach unifies local and global reasoning in transformer layers, resulting in improved quality and faster convergence. The model is trained on large-scale image collections and achieves high-quality reconstruction results using only 4 A100-40G GPUs within a span of 2 days. LaRa's method is efficient and robust, providing photorealistic, 360° novel view synthesis results using only four input images. The model also allows high-quality mesh reconstruction using off-the-shelf depth-map fusion algorithms. LaRa's approach is designed to handle unstructured views and is flexible to accommodate various numbers of views. The model's performance is evaluated on multiple datasets, including the Objaverse dataset, Google Scanned Objects dataset, and the Co3D test set. LaRa outperforms previous methods in terms of rendering quality, geometry accuracy, and generalization to real-world data. The model's effectiveness is demonstrated through quantitative results and visual comparisons with other methods. LaRa's approach is efficient and robust, providing high-quality 3D reconstructions with minimal training resources. The model's performance is further validated through ablation studies and comparisons with other methods. LaRa's method is a significant advancement in the field of radiance field reconstruction, offering a novel approach to large-baseline 3D reconstruction with high efficiency and accuracy.
Reach us at info@study.space
Understanding LaRa%3A Efficient Large-Baseline Radiance Fields