2 Jun 2024 | Weining Ren, Zihan Zhu, Boyang Sun, Jiaqi Chen, Marc Pollefeys, Songyou Peng
NeRF On-the-go is a method that enables the training of Neural Radiance Fields (NeRFs) for dynamic real-world scenes, effectively removing distractors such as moving objects, pedestrians, and vehicles. Unlike existing methods like NeRF-W and RobustNeRF, which struggle with dynamic scenes, NeRF On-the-go leverages predicted uncertainty maps to efficiently eliminate distractors, resulting in high-fidelity novel view synthesis. The method uses DINOv2 features for robust feature extraction and a structural similarity loss to enhance uncertainty optimization. It also incorporates estimated uncertainty into the NeRF image reconstruction objective using a decoupled training strategy, significantly improving distractor elimination, especially in high-occlusion scenarios. The method demonstrates robustness across various scenes, from indoor to outdoor environments, and can handle varying levels of distractors. Additionally, it accelerates NeRF training up to one order of magnitude compared to RobustNeRF. The method is designed to be a versatile plug-and-play module for effective distractor removal, allowing rapid NeRF training from casually captured images. The approach is evaluated on various datasets, showing significant improvements over state-of-the-art techniques. The method's effectiveness is demonstrated through comprehensive experiments, highlighting its potential for diverse and dynamic real-world applications.NeRF On-the-go is a method that enables the training of Neural Radiance Fields (NeRFs) for dynamic real-world scenes, effectively removing distractors such as moving objects, pedestrians, and vehicles. Unlike existing methods like NeRF-W and RobustNeRF, which struggle with dynamic scenes, NeRF On-the-go leverages predicted uncertainty maps to efficiently eliminate distractors, resulting in high-fidelity novel view synthesis. The method uses DINOv2 features for robust feature extraction and a structural similarity loss to enhance uncertainty optimization. It also incorporates estimated uncertainty into the NeRF image reconstruction objective using a decoupled training strategy, significantly improving distractor elimination, especially in high-occlusion scenarios. The method demonstrates robustness across various scenes, from indoor to outdoor environments, and can handle varying levels of distractors. Additionally, it accelerates NeRF training up to one order of magnitude compared to RobustNeRF. The method is designed to be a versatile plug-and-play module for effective distractor removal, allowing rapid NeRF training from casually captured images. The approach is evaluated on various datasets, showing significant improvements over state-of-the-art techniques. The method's effectiveness is demonstrated through comprehensive experiments, highlighting its potential for diverse and dynamic real-world applications.