18 Mar 2024 | Jiaxiang Tang, Ruijie Lu, Xiaokang Chen, Xiang Wen, Gang Zeng, and Ziwei Liu
InteX is an interactive text-to-texture synthesis framework that addresses the challenges of 3D inconsistency and limited controllability in existing methods. The framework includes a user-friendly interface for visualization, inpainting, erasing, and repainting, enabling precise texture editing. It also features a unified depth-aware inpainting model that integrates depth information with inpainting cues, effectively reducing 3D inconsistencies and improving generation speed. Through extensive experiments, the framework has proven to be practical and effective in text-to-texture synthesis, paving the way for high-quality 3D content creation.
The framework leverages a unified depth-aware inpainting prior model trained on 3D datasets, which helps streamline the pipeline and enhance performance. It also employs an iterative texture synthesis algorithm using a 2D prior for efficient texture synthesis and a graphic user interface for interactive viewpoint selection and flexible repainting. The method supports interactive editing, allowing users to repaint specific regions while keeping others unchanged, enhancing controllability and flexibility.
Compared to prior techniques, InteX offers enhanced controllability, efficiency, and flexibility in text-to-texture synthesis. The method achieves 10 times acceleration over previous iterative inpainting-based methods using 10 cameras, while also producing more preferred texture quality with a user-friendly GUI. The framework also demonstrates superior performance in generating textures with enhanced detail and improved 3D consistency, as shown in qualitative comparisons with recent methods.
The method is evaluated through extensive experiments, including quantitative comparisons with state-of-the-art methods and ablation studies on the effect of different 2D diffusion prior models. The results show that InteX achieves higher quality and better 3D consistency in texture generation, with a user study indicating higher satisfaction with the generated textures. The framework is also compared with other methods in terms of generation time and user study results, demonstrating its effectiveness and efficiency in text-to-texture synthesis.InteX is an interactive text-to-texture synthesis framework that addresses the challenges of 3D inconsistency and limited controllability in existing methods. The framework includes a user-friendly interface for visualization, inpainting, erasing, and repainting, enabling precise texture editing. It also features a unified depth-aware inpainting model that integrates depth information with inpainting cues, effectively reducing 3D inconsistencies and improving generation speed. Through extensive experiments, the framework has proven to be practical and effective in text-to-texture synthesis, paving the way for high-quality 3D content creation.
The framework leverages a unified depth-aware inpainting prior model trained on 3D datasets, which helps streamline the pipeline and enhance performance. It also employs an iterative texture synthesis algorithm using a 2D prior for efficient texture synthesis and a graphic user interface for interactive viewpoint selection and flexible repainting. The method supports interactive editing, allowing users to repaint specific regions while keeping others unchanged, enhancing controllability and flexibility.
Compared to prior techniques, InteX offers enhanced controllability, efficiency, and flexibility in text-to-texture synthesis. The method achieves 10 times acceleration over previous iterative inpainting-based methods using 10 cameras, while also producing more preferred texture quality with a user-friendly GUI. The framework also demonstrates superior performance in generating textures with enhanced detail and improved 3D consistency, as shown in qualitative comparisons with recent methods.
The method is evaluated through extensive experiments, including quantitative comparisons with state-of-the-art methods and ablation studies on the effect of different 2D diffusion prior models. The results show that InteX achieves higher quality and better 3D consistency in texture generation, with a user study indicating higher satisfaction with the generated textures. The framework is also compared with other methods in terms of generation time and user study results, demonstrating its effectiveness and efficiency in text-to-texture synthesis.