17 May 2024 | Hongkun Zhang, Zherong Pan, Congyi Zhang, Lifeng Zhu, and Xifeng Gao
**TexPainter: Generative Mesh Texturing with Multi-view Consistency**
**Authors:** Hongkun Zhang, Zherong Pan, Congyi Zhang
**Abstract:**
This paper addresses the challenge of generating high-quality, multi-view consistent textures for 3D models using pre-trained diffusion models. Traditional methods often struggle with multi-view consistency, leading to suboptimal results. TexPainter proposes a novel approach that leverages the Denoising Diffusion Implicit Models (DDIM) scheme to enforce multi-view consistency. By optimizing the latent codes in color space, TexPainter avoids direct manipulation of latent codes and relaxes the sequential dependency assumption among camera views. Extensive experiments on a variety of 3D models demonstrate that TexPainter significantly improves texture quality and consistency compared to state-of-the-art methods.
**Key Contributions:**
1. **Multi-view Consistency Enforcement:** TexPainter uses DDIM to predict noiseless latent states, which are then decoded into color space for multi-view fusion. This approach ensures that the generated texture is consistent across all camera views.
2. **Optimization-Based Color Fusion:** The method updates the latent codes by optimizing the color space texture, eliminating the need for sequential denoising processes.
3. **Relaxation of Sequential Dependency:** TexPainter does not assume sequential dependence between camera views, making it more flexible and effective.
**Evaluation:**
TexPainter is evaluated on a set of 64 high-quality 3D models, with 182 pairs of meshes and text prompts. Results show that TexPainter consistently generates high-quality textures with better multi-view consistency compared to competing methods such as TEXTure, TexFusion, Meshy, and Fantasia3D.
**Conclusion:**
TexPainter offers a novel approach to generate high-quality, multi-view consistent textures for 3D models using pre-trained diffusion models. By leveraging DDIM and optimization-based color fusion, TexPainter achieves superior texture quality and consistency, making it a promising tool for 3D content creation.**TexPainter: Generative Mesh Texturing with Multi-view Consistency**
**Authors:** Hongkun Zhang, Zherong Pan, Congyi Zhang
**Abstract:**
This paper addresses the challenge of generating high-quality, multi-view consistent textures for 3D models using pre-trained diffusion models. Traditional methods often struggle with multi-view consistency, leading to suboptimal results. TexPainter proposes a novel approach that leverages the Denoising Diffusion Implicit Models (DDIM) scheme to enforce multi-view consistency. By optimizing the latent codes in color space, TexPainter avoids direct manipulation of latent codes and relaxes the sequential dependency assumption among camera views. Extensive experiments on a variety of 3D models demonstrate that TexPainter significantly improves texture quality and consistency compared to state-of-the-art methods.
**Key Contributions:**
1. **Multi-view Consistency Enforcement:** TexPainter uses DDIM to predict noiseless latent states, which are then decoded into color space for multi-view fusion. This approach ensures that the generated texture is consistent across all camera views.
2. **Optimization-Based Color Fusion:** The method updates the latent codes by optimizing the color space texture, eliminating the need for sequential denoising processes.
3. **Relaxation of Sequential Dependency:** TexPainter does not assume sequential dependence between camera views, making it more flexible and effective.
**Evaluation:**
TexPainter is evaluated on a set of 64 high-quality 3D models, with 182 pairs of meshes and text prompts. Results show that TexPainter consistently generates high-quality textures with better multi-view consistency compared to competing methods such as TEXTure, TexFusion, Meshy, and Fantasia3D.
**Conclusion:**
TexPainter offers a novel approach to generate high-quality, multi-view consistent textures for 3D models using pre-trained diffusion models. By leveraging DDIM and optimization-based color fusion, TexPainter achieves superior texture quality and consistency, making it a promising tool for 3D content creation.