FreeSplat: Generalizable 3D Gaussian Splatting Towards Free-View Synthesis of Indoor Scenes

FreeSplat: Generalizable 3D Gaussian Splatting Towards Free-View Synthesis of Indoor Scenes

9 Jun 2024 | Yunsong Wang, Tianxin Huang, Hanlin Chen, Gim Hee Lee
FreeSplat is a novel framework for generalizable 3D Gaussian Splatting that enables free-view synthesis of indoor scenes from long input sequences. Unlike previous methods that are limited to narrow-range view interpolation, FreeSplat can accurately localize 3D Gaussians across wide view ranges, supporting free-view synthesis. The framework consists of two main components: Low-cost Cross-View Aggregation and Pixel-wise Triplet Fusion (PTF). Low-cost Cross-View Aggregation efficiently extracts and matches features across nearby views, while PTF fuses local Gaussian triplets to reduce redundancy and aggregate multi-view features. FreeSplat also introduces a Free-View Training (FVT) strategy that enables robust view synthesis across broader view ranges regardless of the number of input views. Empirical results show that FreeSplat achieves state-of-the-art performance in novel view synthesis, with high-quality color maps and accurate depth maps across different input view lengths. Additionally, FreeSplat is efficient in inference and can reduce redundant Gaussians, enabling feed-forward large scene reconstruction without depth priors. The code will be made open-source upon paper acceptance.FreeSplat is a novel framework for generalizable 3D Gaussian Splatting that enables free-view synthesis of indoor scenes from long input sequences. Unlike previous methods that are limited to narrow-range view interpolation, FreeSplat can accurately localize 3D Gaussians across wide view ranges, supporting free-view synthesis. The framework consists of two main components: Low-cost Cross-View Aggregation and Pixel-wise Triplet Fusion (PTF). Low-cost Cross-View Aggregation efficiently extracts and matches features across nearby views, while PTF fuses local Gaussian triplets to reduce redundancy and aggregate multi-view features. FreeSplat also introduces a Free-View Training (FVT) strategy that enables robust view synthesis across broader view ranges regardless of the number of input views. Empirical results show that FreeSplat achieves state-of-the-art performance in novel view synthesis, with high-quality color maps and accurate depth maps across different input view lengths. Additionally, FreeSplat is efficient in inference and can reduce redundant Gaussians, enabling feed-forward large scene reconstruction without depth priors. The code will be made open-source upon paper acceptance.
Reach us at info@study.space