23 Mar 2020 | Michael Niemeyer, Lars Mescheder, Michael Oechsle, Andreas Geiger
This paper introduces Differentiable Volumetric Rendering (DVR), a novel approach to learning implicit 3D representations from RGB images without requiring 3D supervision. The key insight is that depth gradients can be derived analytically using implicit differentiation, allowing the model to learn implicit shape and texture representations directly from 2D images. The method is memory-efficient as it does not need to store volumetric data during the forward pass. Experiments show that DVR can achieve competitive results with full 3D supervision in single-view reconstruction tasks and can also be used for multi-view 3D reconstruction, producing watertight meshes. The method is validated on various datasets and compared against several state-of-the-art methods, demonstrating its effectiveness and versatility.This paper introduces Differentiable Volumetric Rendering (DVR), a novel approach to learning implicit 3D representations from RGB images without requiring 3D supervision. The key insight is that depth gradients can be derived analytically using implicit differentiation, allowing the model to learn implicit shape and texture representations directly from 2D images. The method is memory-efficient as it does not need to store volumetric data during the forward pass. Experiments show that DVR can achieve competitive results with full 3D supervision in single-view reconstruction tasks and can also be used for multi-view 3D reconstruction, producing watertight meshes. The method is validated on various datasets and compared against several state-of-the-art methods, demonstrating its effectiveness and versatility.