TextureDreamer: Image-guided Texture Synthesis through Geometry-aware Diffusion

TextureDreamer: Image-guided Texture Synthesis through Geometry-aware Diffusion

17 Jan 2024 | Yu-Ying Yeh13 Jia-Bin Huang23 Changil Kim3 Lei Xiao3 Thu Nguyen-Phuoc3 Numair Khan3 Cheng Zhang3 Manmohan Chandraker1 Carl S Marshall3 Zhao Dong3 Zhengqin Li3
TextureDreamer is a novel image-guided texture synthesis method that transfers relightable textures from a small number of input images (3 to 5) to target 3D shapes across various categories. The method, inspired by recent advancements in diffuse models, includes personalized geometry-aware score distillation (PGSD), which extracts texture information from sparse images and aligns it with the target geometry. PGSD uses variational score distillation to generate more photorealistic and diverse textures, and explicit geometry guidance with ControlNet to ensure 3D consistency. Experiments on real images show that TextureDreamer can successfully transfer highly realistic and semantically meaningful textures to arbitrary objects, outperforming previous state-of-the-art methods. The framework is evaluated through user studies and quantitative metrics, demonstrating its effectiveness in texture transfer and 3D content creation.TextureDreamer is a novel image-guided texture synthesis method that transfers relightable textures from a small number of input images (3 to 5) to target 3D shapes across various categories. The method, inspired by recent advancements in diffuse models, includes personalized geometry-aware score distillation (PGSD), which extracts texture information from sparse images and aligns it with the target geometry. PGSD uses variational score distillation to generate more photorealistic and diverse textures, and explicit geometry guidance with ControlNet to ensure 3D consistency. Experiments on real images show that TextureDreamer can successfully transfer highly realistic and semantically meaningful textures to arbitrary objects, outperforming previous state-of-the-art methods. The framework is evaluated through user studies and quantitative metrics, demonstrating its effectiveness in texture transfer and 3D content creation.
Reach us at info@study.space
[slides and audio] TextureDreamer%3A Image-Guided Texture Synthesis through Geometry-Aware Diffusion