Good Seed Makes a Good Crop: Discovering Secret Seeds in Text-to-Image Diffusion Models

Good Seed Makes a Good Crop: Discovering Secret Seeds in Text-to-Image Diffusion Models

23 May 2024 | Katherine Xu, Lingzhi Zhang, Jianbo Shi
This paper explores the impact of random seeds on text-to-image (T2I) diffusion models, revealing that the choice of seed significantly influences the quality, style, and composition of generated images. The study uses a large-scale dataset generated from two T2I models, SD 2.0 and SDXL Turbo, with over 46 million images. Key findings include: 1. **Seed Differentiability**: A classifier trained to predict the seed number used to generate an image achieved over 99.9% accuracy, demonstrating that seeds are highly distinguishable based on the generated images. 2. **Image Quality and Style**: The best 'golden' seed produced images with an FID score of 21.60, while the worst 'inferior' seed scored 31.97. Seeds also influence image style, with certain seeds consistently generating grayscale images, prominent sky regions, or image borders. 3. **Image Composition**: Seeds affect object location, size, and depth, showing consistent patterns across different prompts. 4. **Practical Applications**: The paper proposes downstream applications such as high-fidelity inference and diversified sampling using 'golden' seeds. For example, leveraging these seeds can improve image quality and human preference alignment. 5. **Text-based Inpainting**: Some seeds tend to insert unwanted text artifacts during inpainting tasks, which can be mitigated by selecting appropriate seeds. Overall, the study highlights the importance of selecting good seeds to enhance image generation and offers practical utility for improving T2I models without significant computational overhead.This paper explores the impact of random seeds on text-to-image (T2I) diffusion models, revealing that the choice of seed significantly influences the quality, style, and composition of generated images. The study uses a large-scale dataset generated from two T2I models, SD 2.0 and SDXL Turbo, with over 46 million images. Key findings include: 1. **Seed Differentiability**: A classifier trained to predict the seed number used to generate an image achieved over 99.9% accuracy, demonstrating that seeds are highly distinguishable based on the generated images. 2. **Image Quality and Style**: The best 'golden' seed produced images with an FID score of 21.60, while the worst 'inferior' seed scored 31.97. Seeds also influence image style, with certain seeds consistently generating grayscale images, prominent sky regions, or image borders. 3. **Image Composition**: Seeds affect object location, size, and depth, showing consistent patterns across different prompts. 4. **Practical Applications**: The paper proposes downstream applications such as high-fidelity inference and diversified sampling using 'golden' seeds. For example, leveraging these seeds can improve image quality and human preference alignment. 5. **Text-based Inpainting**: Some seeds tend to insert unwanted text artifacts during inpainting tasks, which can be mitigated by selecting appropriate seeds. Overall, the study highlights the importance of selecting good seeds to enhance image generation and offers practical utility for improving T2I models without significant computational overhead.
Reach us at info@study.space