17 Aug 2021 | Alex Yu1 Ruilong Li1,2 Matthew Tancik1 Hao Li1,3 Ren Ng1 Angjoo Kanazawa1
The paper introduces PlenOctrees, a novel data structure for real-time rendering of Neural Radiance Fields (NeRFs). PlenOctrees are octrees that capture view-dependent effects, enabling high-quality rendering at over 150 FPS, which is more than 3000 times faster than conventional NeRFs. The method pre-tabulates the NeRF into a PlenOctree, allowing for efficient rendering without sacrificing quality. To preserve view-dependent effects like specularities, the appearance is factorized using spherical basis functions. The NeRF network is modified to predict spherical harmonic coefficients, eliminating the need for viewing direction input. PlenOctrees can be optimized directly, improving reconstruction loss and quality. This optimization step also reduces training time. The approach enables applications such as 6-DOF industrial and product visualizations, and AR/VR systems. The paper includes experimental results demonstrating the effectiveness of PlenOctrees in terms of speed and image quality.The paper introduces PlenOctrees, a novel data structure for real-time rendering of Neural Radiance Fields (NeRFs). PlenOctrees are octrees that capture view-dependent effects, enabling high-quality rendering at over 150 FPS, which is more than 3000 times faster than conventional NeRFs. The method pre-tabulates the NeRF into a PlenOctree, allowing for efficient rendering without sacrificing quality. To preserve view-dependent effects like specularities, the appearance is factorized using spherical basis functions. The NeRF network is modified to predict spherical harmonic coefficients, eliminating the need for viewing direction input. PlenOctrees can be optimized directly, improving reconstruction loss and quality. This optimization step also reduces training time. The approach enables applications such as 6-DOF industrial and product visualizations, and AR/VR systems. The paper includes experimental results demonstrating the effectiveness of PlenOctrees in terms of speed and image quality.