1 Apr 2024 | Zizhang Li, Dor Litvak, Ruining Li, Yunzhi Zhang, Tomas Jakab, Christian Rupprecht, Shangzhe Wu, Andrea Vedaldi, Jiajun Wu
The paper introduces 3D-Fauna, a method that learns a pan-category deformable 3D model of over 100 different animal species using only 2D Internet images as training data. The method overcomes the limited availability of training data by leveraging a Semantic Bank of Skinned Models (SBSM), which automatically discovers a small set of base animal shapes by combining geometric inductive priors with semantic knowledge implicit in off-the-shelf self-supervised feature extractors. The model can reconstruct an articulated, textured 3D mesh from a single image of any quadruped animal in a feed-forward manner, ready for animation and rendering. The paper also presents a large-scale dataset of diverse animal species, the Fauna Dataset, which includes over 100 quadruped species. Extensive quantitative and qualitative comparisons demonstrate significant improvements over existing methods.The paper introduces 3D-Fauna, a method that learns a pan-category deformable 3D model of over 100 different animal species using only 2D Internet images as training data. The method overcomes the limited availability of training data by leveraging a Semantic Bank of Skinned Models (SBSM), which automatically discovers a small set of base animal shapes by combining geometric inductive priors with semantic knowledge implicit in off-the-shelf self-supervised feature extractors. The model can reconstruct an articulated, textured 3D mesh from a single image of any quadruped animal in a feed-forward manner, ready for animation and rendering. The paper also presents a large-scale dataset of diverse animal species, the Fauna Dataset, which includes over 100 quadruped species. Extensive quantitative and qualitative comparisons demonstrate significant improvements over existing methods.