TexPainter: Generative Mesh Texturing with Multi-view Consistency

TexPainter: Generative Mesh Texturing with Multi-view Consistency

17 May 2024 | Hongkun Zhang, Zherong Pan, Congyi Zhang, Lifeng Zhu, and Xifeng Gao
**TexPainter: Generative Mesh Texturing with Multi-view Consistency** **Authors:** Hongkun Zhang, Zherong Pan, Congyi Zhang **Abstract:** This paper addresses the challenge of generating high-quality, multi-view consistent textures for 3D models using pre-trained diffusion models. Traditional methods often struggle with multi-view consistency, leading to suboptimal results. TexPainter proposes a novel approach that leverages the Denoising Diffusion Implicit Models (DDIM) scheme to enforce multi-view consistency. By optimizing the latent codes in color space, TexPainter avoids direct manipulation of latent codes and relaxes the sequential dependency assumption among camera views. Extensive experiments on a variety of 3D models demonstrate that TexPainter significantly improves texture quality and consistency compared to state-of-the-art methods. **Key Contributions:** 1. **Multi-view Consistency Enforcement:** TexPainter uses DDIM to predict noiseless latent states, which are then decoded into color space for multi-view fusion. This approach ensures that the generated texture is consistent across all camera views. 2. **Optimization-Based Color Fusion:** The method updates the latent codes by optimizing the color space texture, eliminating the need for sequential denoising processes. 3. **Relaxation of Sequential Dependency:** TexPainter does not assume sequential dependence between camera views, making it more flexible and effective. **Evaluation:** TexPainter is evaluated on a set of 64 high-quality 3D models, with 182 pairs of meshes and text prompts. Results show that TexPainter consistently generates high-quality textures with better multi-view consistency compared to competing methods such as TEXTure, TexFusion, Meshy, and Fantasia3D. **Conclusion:** TexPainter offers a novel approach to generate high-quality, multi-view consistent textures for 3D models using pre-trained diffusion models. By leveraging DDIM and optimization-based color fusion, TexPainter achieves superior texture quality and consistency, making it a promising tool for 3D content creation.**TexPainter: Generative Mesh Texturing with Multi-view Consistency** **Authors:** Hongkun Zhang, Zherong Pan, Congyi Zhang **Abstract:** This paper addresses the challenge of generating high-quality, multi-view consistent textures for 3D models using pre-trained diffusion models. Traditional methods often struggle with multi-view consistency, leading to suboptimal results. TexPainter proposes a novel approach that leverages the Denoising Diffusion Implicit Models (DDIM) scheme to enforce multi-view consistency. By optimizing the latent codes in color space, TexPainter avoids direct manipulation of latent codes and relaxes the sequential dependency assumption among camera views. Extensive experiments on a variety of 3D models demonstrate that TexPainter significantly improves texture quality and consistency compared to state-of-the-art methods. **Key Contributions:** 1. **Multi-view Consistency Enforcement:** TexPainter uses DDIM to predict noiseless latent states, which are then decoded into color space for multi-view fusion. This approach ensures that the generated texture is consistent across all camera views. 2. **Optimization-Based Color Fusion:** The method updates the latent codes by optimizing the color space texture, eliminating the need for sequential denoising processes. 3. **Relaxation of Sequential Dependency:** TexPainter does not assume sequential dependence between camera views, making it more flexible and effective. **Evaluation:** TexPainter is evaluated on a set of 64 high-quality 3D models, with 182 pairs of meshes and text prompts. Results show that TexPainter consistently generates high-quality textures with better multi-view consistency compared to competing methods such as TEXTure, TexFusion, Meshy, and Fantasia3D. **Conclusion:** TexPainter offers a novel approach to generate high-quality, multi-view consistent textures for 3D models using pre-trained diffusion models. By leveraging DDIM and optimization-based color fusion, TexPainter achieves superior texture quality and consistency, making it a promising tool for 3D content creation.
Reach us at info@study.space
[slides and audio] TexPainter%3A Generative Mesh Texturing with Multi-view Consistency