DreamMat is a novel method for generating high-quality PBR materials from text descriptions on untextured 3D meshes. The method addresses the issue of incorrect material decomposition in previous approaches, which often result in baked-in shading effects. DreamMat introduces a geometry- and light-aware diffusion model to generate materials that are consistent with the given geometry and environment light, avoiding incorrect material decomposition. The method first fine-tunes a new light-aware 2D diffusion model to condition on a given lighting environment and generate shading results on this specific lighting condition. Then, by applying the same environment lights in the material distillation, DreamMat can generate high-quality PBR materials that are not only consistent with the given geometry but also free from any baked-in shading effects in albedo. Extensive experiments demonstrate that the materials produced through our methods exhibit greater visual appeal to users and achieve significantly superior rendering quality compared to baseline methods, which are preferable for downstream tasks such as game and film production. The method is based on a geometry- and light-aware diffusion model and an inverse rendering-based distillation method. The inverse rendering method renders images by Monte Carlo sampling and distills materials by a CSD loss. The key advantage of our method is the geometry- and light-aware diffusion model which can generate images consistent with the geometry and environment light. Distilling from this diffusion model avoids the common problem of baking shading effects into albedo. We demonstrate that the generated materials by our method are readily usable in modern graphics engines, offering enhanced realism for various applications in gaming and simulation. The method has several limitations, including the inability to handle materials with properties like transparency, high reflection, or subsurface scattering, and the relatively long time required for high-quality generation. However, the method shows promising results in generating diverse and high-quality appearances according to text prompts.DreamMat is a novel method for generating high-quality PBR materials from text descriptions on untextured 3D meshes. The method addresses the issue of incorrect material decomposition in previous approaches, which often result in baked-in shading effects. DreamMat introduces a geometry- and light-aware diffusion model to generate materials that are consistent with the given geometry and environment light, avoiding incorrect material decomposition. The method first fine-tunes a new light-aware 2D diffusion model to condition on a given lighting environment and generate shading results on this specific lighting condition. Then, by applying the same environment lights in the material distillation, DreamMat can generate high-quality PBR materials that are not only consistent with the given geometry but also free from any baked-in shading effects in albedo. Extensive experiments demonstrate that the materials produced through our methods exhibit greater visual appeal to users and achieve significantly superior rendering quality compared to baseline methods, which are preferable for downstream tasks such as game and film production. The method is based on a geometry- and light-aware diffusion model and an inverse rendering-based distillation method. The inverse rendering method renders images by Monte Carlo sampling and distills materials by a CSD loss. The key advantage of our method is the geometry- and light-aware diffusion model which can generate images consistent with the geometry and environment light. Distilling from this diffusion model avoids the common problem of baking shading effects into albedo. We demonstrate that the generated materials by our method are readily usable in modern graphics engines, offering enhanced realism for various applications in gaming and simulation. The method has several limitations, including the inability to handle materials with properties like transparency, high reflection, or subsurface scattering, and the relatively long time required for high-quality generation. However, the method shows promising results in generating diverse and high-quality appearances according to text prompts.