2 Jul 2024 | Raphael Bensadoun*, Yanir Kleiman*, Idan Azuri, Omri Harosh, Andrea Vedaldi, Natalia Neverova, Oran Gafni
Meta 3D TextureGen is a novel method for generating high-quality, globally consistent textures for 3D objects from textual prompts. The method consists of two sequential networks that work in image space and UV space, respectively. In image space, a fine-tuned text-to-image model generates multi-view images of the textured 3D shape, conditioned on 2D renders of 3D features (position and normal maps). In UV space, an inpainting network completes missing areas and enhances the texture, producing a complete UV texture map. The method achieves state-of-the-art results in quality and speed, with a runtime of under 20 seconds. It addresses key challenges such as global consistency, text faithfulness, and semantic alignment with the target 3D shape. An additional texture enhancement network can up-scale the generated textures to 4k resolution. Extensive qualitative and quantitative evaluations demonstrate the effectiveness of Meta 3D TextureGen, showing superior performance over existing methods in terms of visual fidelity, text alignment, and artifact reduction.Meta 3D TextureGen is a novel method for generating high-quality, globally consistent textures for 3D objects from textual prompts. The method consists of two sequential networks that work in image space and UV space, respectively. In image space, a fine-tuned text-to-image model generates multi-view images of the textured 3D shape, conditioned on 2D renders of 3D features (position and normal maps). In UV space, an inpainting network completes missing areas and enhances the texture, producing a complete UV texture map. The method achieves state-of-the-art results in quality and speed, with a runtime of under 20 seconds. It addresses key challenges such as global consistency, text faithfulness, and semantic alignment with the target 3D shape. An additional texture enhancement network can up-scale the generated textures to 4k resolution. Extensive qualitative and quantitative evaluations demonstrate the effectiveness of Meta 3D TextureGen, showing superior performance over existing methods in terms of visual fidelity, text alignment, and artifact reduction.