The paper "Collaborative Control for Geometry-Conditioned PBR Image Generation" addresses the challenge of generating physically-based rendering (PBR) images that are conditioned on 3D geometry and prompts. The authors propose a novel approach called *Collaborative Control*, which involves training a PBR diffusion model in tandem with a pre-trained RGB image model. The PBR model is linked to the frozen RGB model through a cross-network communication paradigm, allowing the PBR model to leverage the rich internal state of the RGB model while maintaining its general performance and compatibility with techniques like IPAdapters. Key contributions of the paper include: 1. **Collaborative Control**: A new paradigm that enables bidirectional communication between the PBR and RGB models, ensuring that the PBR model can extract relevant information from the RGB model while guiding the RGB output towards render-like images. 2. **Data Efficiency**: The proposed method is highly data-efficient, even with limited training data, and performs well on out-of-distribution (OOD) prompts. 3. **Compatibility with Existing Techniques**: The method is compatible with existing control techniques, such as IP-Adapter, which allows for style guidance in PBR content generation. The paper evaluates the proposed method through various experiments, including comparisons with other control paradigms, communication types, and fine-tuning approaches. It also demonstrates the method's ability to generate high-quality PBR images, including relighting and interpolation, and discusses limitations and failure cases. The results show that Collaborative Control effectively addresses the challenges of PBR image generation, opening up new avenues for graphics applications, particularly in Text-to-Texture.The paper "Collaborative Control for Geometry-Conditioned PBR Image Generation" addresses the challenge of generating physically-based rendering (PBR) images that are conditioned on 3D geometry and prompts. The authors propose a novel approach called *Collaborative Control*, which involves training a PBR diffusion model in tandem with a pre-trained RGB image model. The PBR model is linked to the frozen RGB model through a cross-network communication paradigm, allowing the PBR model to leverage the rich internal state of the RGB model while maintaining its general performance and compatibility with techniques like IPAdapters. Key contributions of the paper include: 1. **Collaborative Control**: A new paradigm that enables bidirectional communication between the PBR and RGB models, ensuring that the PBR model can extract relevant information from the RGB model while guiding the RGB output towards render-like images. 2. **Data Efficiency**: The proposed method is highly data-efficient, even with limited training data, and performs well on out-of-distribution (OOD) prompts. 3. **Compatibility with Existing Techniques**: The method is compatible with existing control techniques, such as IP-Adapter, which allows for style guidance in PBR content generation. The paper evaluates the proposed method through various experiments, including comparisons with other control paradigms, communication types, and fine-tuning approaches. It also demonstrates the method's ability to generate high-quality PBR images, including relighting and interpolation, and discusses limitations and failure cases. The results show that Collaborative Control effectively addresses the challenges of PBR image generation, opening up new avenues for graphics applications, particularly in Text-to-Texture.