NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections

NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections

6 Jan 2021 | Ricardo Martin-Brualla*, Noha Radwan*, Mehdi S. M. Sajjadi*, Jonathan T. Barron, Alexey Dosovitskiy, and Daniel Duckworth
The paper presents NeRF-W, a method for synthesizing novel views of complex scenes using unstructured collections of in-the-wild photographs. Building on Neural Radiance Fields (NeRF), which models scene density and color as a function of 3D coordinates, NeRF-W addresses limitations of NeRF by relaxing its strict consistency assumptions. NeRF-W introduces two key extensions: (1) a learned low-dimensional latent space to model per-image appearance variations such as exposure, lighting, and post-processing, and (2) a decomposition of the scene into shared and image-dependent elements to handle transient objects. These extensions enable the model to disentangle static and transient components, allowing for accurate reconstructions from unstructured image collections. The system is applied to internet photo collections of famous landmarks, demonstrating temporally consistent and photorealistic novel view renderings that surpass previous state-of-the-art methods in terms of PSNR and MS-SSIM metrics. NeRF-W shows improved quality over NeRF in the presence of appearance variation and transient occluders while achieving similar results in controlled settings.The paper presents NeRF-W, a method for synthesizing novel views of complex scenes using unstructured collections of in-the-wild photographs. Building on Neural Radiance Fields (NeRF), which models scene density and color as a function of 3D coordinates, NeRF-W addresses limitations of NeRF by relaxing its strict consistency assumptions. NeRF-W introduces two key extensions: (1) a learned low-dimensional latent space to model per-image appearance variations such as exposure, lighting, and post-processing, and (2) a decomposition of the scene into shared and image-dependent elements to handle transient objects. These extensions enable the model to disentangle static and transient components, allowing for accurate reconstructions from unstructured image collections. The system is applied to internet photo collections of famous landmarks, demonstrating temporally consistent and photorealistic novel view renderings that surpass previous state-of-the-art methods in terms of PSNR and MS-SSIM metrics. NeRF-W shows improved quality over NeRF in the presence of appearance variation and transient occluders while achieving similar results in controlled settings.
Reach us at info@study.space