6 Jan 2021 | Lingjie Liu†*, Jiatao Gu†*, Kyaw Zaw Lin†, Tat-Seng Chua°, Christian Theobalt†
Neural Sparse Voxel Fields (NSVF) is a new neural scene representation for fast and high-quality free-viewpoint rendering. Unlike traditional methods that require 3D supervision, NSVF learns scene representations implicitly by using a sparse voxel octree structure. This structure allows for efficient rendering by skipping voxels that do not contain relevant scene content. NSVF is trained using a differentiable ray-marching operation from a set of posed RGB images, enabling it to progressively learn the underlying voxel structure. The method is significantly faster than state-of-the-art approaches like NeRF while achieving higher quality results. NSVF can be easily applied to scene editing and composition, and it supports challenging tasks such as multi-scene learning, free-viewpoint rendering of moving humans, and large-scale scene rendering. The method is efficient in terms of both computation and storage, with a storage footprint that is significantly smaller than that of NeRF. Nsfv is able to render high-quality images at a much faster rate than traditional methods, making it suitable for real-time applications. The method is also able to handle complex scenes with intricate geometry and lighting effects, demonstrating its versatility and effectiveness in a wide range of applications.Neural Sparse Voxel Fields (NSVF) is a new neural scene representation for fast and high-quality free-viewpoint rendering. Unlike traditional methods that require 3D supervision, NSVF learns scene representations implicitly by using a sparse voxel octree structure. This structure allows for efficient rendering by skipping voxels that do not contain relevant scene content. NSVF is trained using a differentiable ray-marching operation from a set of posed RGB images, enabling it to progressively learn the underlying voxel structure. The method is significantly faster than state-of-the-art approaches like NeRF while achieving higher quality results. NSVF can be easily applied to scene editing and composition, and it supports challenging tasks such as multi-scene learning, free-viewpoint rendering of moving humans, and large-scale scene rendering. The method is efficient in terms of both computation and storage, with a storage footprint that is significantly smaller than that of NeRF. Nsfv is able to render high-quality images at a much faster rate than traditional methods, making it suitable for real-time applications. The method is also able to handle complex scenes with intricate geometry and lighting effects, demonstrating its versatility and effectiveness in a wide range of applications.