29 Nov 2022 | Anpei Chen, Zexiang Xu, Andreas Geiger, Jingyi Yu, Hao Su
TensoRF is a novel approach for modeling and reconstructing radiance fields, which represents scenes as 4D tensors instead of using MLPs as in NeRF. The 4D tensor represents a 3D voxel grid with per-voxel multi-channel features. The key idea is to factorize the 4D scene tensor into multiple compact low-rank tensor components. This approach leads to improved rendering quality and significantly lower memory footprint compared to previous methods. Two decomposition techniques are introduced: CP decomposition and vector-matrix (VM) decomposition. CP decomposition factorizes the tensor into rank-one components, while VM decomposition relaxes the low-rank constraints for two modes of a tensor and factorizes it into compact vector and matrix factors. TensoRF with CP decomposition achieves fast reconstruction (<30 min) with better rendering quality and smaller model size (<4 MB) compared to NeRF. TensoRF with VM decomposition further improves rendering quality and reduces reconstruction time (<10 min) while maintaining a compact model size (<75 MB). The approach enables efficient scene reconstruction and modeling, supporting various types of per-voxel features with different decoding functions. The tensorial radiance fields can be effectively reconstructed from multi-view images and enable realistic novel view synthesis. The approach is efficient in both training time and memory footprint, achieving high computation and memory efficiency. TensoRF models can reconstruct high-quality radiance fields in 30 min, with the fastest model with VM decomposition taking less than 10 min, significantly faster than NeRF and many other methods. The approach is the first to view radiance field modeling from a tensorial perspective and pose the problem of radiance field reconstruction as one of low-rank tensor reconstructions.TensoRF is a novel approach for modeling and reconstructing radiance fields, which represents scenes as 4D tensors instead of using MLPs as in NeRF. The 4D tensor represents a 3D voxel grid with per-voxel multi-channel features. The key idea is to factorize the 4D scene tensor into multiple compact low-rank tensor components. This approach leads to improved rendering quality and significantly lower memory footprint compared to previous methods. Two decomposition techniques are introduced: CP decomposition and vector-matrix (VM) decomposition. CP decomposition factorizes the tensor into rank-one components, while VM decomposition relaxes the low-rank constraints for two modes of a tensor and factorizes it into compact vector and matrix factors. TensoRF with CP decomposition achieves fast reconstruction (<30 min) with better rendering quality and smaller model size (<4 MB) compared to NeRF. TensoRF with VM decomposition further improves rendering quality and reduces reconstruction time (<10 min) while maintaining a compact model size (<75 MB). The approach enables efficient scene reconstruction and modeling, supporting various types of per-voxel features with different decoding functions. The tensorial radiance fields can be effectively reconstructed from multi-view images and enable realistic novel view synthesis. The approach is efficient in both training time and memory footprint, achieving high computation and memory efficiency. TensoRF models can reconstruct high-quality radiance fields in 30 min, with the fastest model with VM decomposition taking less than 10 min, significantly faster than NeRF and many other methods. The approach is the first to view radiance field modeling from a tensorial perspective and pose the problem of radiance field reconstruction as one of low-rank tensor reconstructions.