AIGIQA-20K is a large-scale database for AI-generated image quality assessment, containing 20,000 AI-generated images (AIGIs) and 420,000 subjective ratings. It was created through a challenge organized by the NTIRE 2024, which evaluated the quality of 15 popular generative models under various dynamic hyper-parameters, including classifier-free guidance, iteration epochs, and output resolution. The database includes comprehensive subjective scores that consider both perceptual quality and text-to-image alignment, collected from 21 subjects. This database is the largest fine-grained AIGI subjective quality database to date.
The database was constructed by dynamically adjusting hyper-parameters for each T2I model, including iterations, CFG, and resolution. It includes 20,000 images generated from 15 representative T2I models, with different configurations to reflect the actual distortion of AIGIs. The prompts used in the database were selected from real user inputs, ensuring diversity and relevance to real-world usage scenarios. The database also includes detailed analysis of quality-related attributes such as light, contrast, color, blur, and spatial information.
Subjective experiments were conducted with 21 participants to evaluate the quality of AIGIs, with scores ranging from 0 to 5. The data was processed to normalize scores and convert them to Z-scores for more accurate analysis. The results showed that the quality of AIGIs is influenced by factors such as the T2I model, prompts, and hyper-parameters. The database was used to benchmark 16 mainstream AIGI quality models, including both perception and alignment metrics. The results indicated that fine-tuning is important for AIGC quality assessment, and that the performance of these models can be significantly improved with proper training.
The database provides a comprehensive resource for evaluating the quality of AI-generated images, and is expected to inspire robust quality indicators for AIGIs and promote the development of AIGC for vision. The database is available for download at https://www.modelscope.cn/datasets/lcysyzxdxc/AIGCQA-30K-Image.AIGIQA-20K is a large-scale database for AI-generated image quality assessment, containing 20,000 AI-generated images (AIGIs) and 420,000 subjective ratings. It was created through a challenge organized by the NTIRE 2024, which evaluated the quality of 15 popular generative models under various dynamic hyper-parameters, including classifier-free guidance, iteration epochs, and output resolution. The database includes comprehensive subjective scores that consider both perceptual quality and text-to-image alignment, collected from 21 subjects. This database is the largest fine-grained AIGI subjective quality database to date.
The database was constructed by dynamically adjusting hyper-parameters for each T2I model, including iterations, CFG, and resolution. It includes 20,000 images generated from 15 representative T2I models, with different configurations to reflect the actual distortion of AIGIs. The prompts used in the database were selected from real user inputs, ensuring diversity and relevance to real-world usage scenarios. The database also includes detailed analysis of quality-related attributes such as light, contrast, color, blur, and spatial information.
Subjective experiments were conducted with 21 participants to evaluate the quality of AIGIs, with scores ranging from 0 to 5. The data was processed to normalize scores and convert them to Z-scores for more accurate analysis. The results showed that the quality of AIGIs is influenced by factors such as the T2I model, prompts, and hyper-parameters. The database was used to benchmark 16 mainstream AIGI quality models, including both perception and alignment metrics. The results indicated that fine-tuning is important for AIGC quality assessment, and that the performance of these models can be significantly improved with proper training.
The database provides a comprehensive resource for evaluating the quality of AI-generated images, and is expected to inspire robust quality indicators for AIGIs and promote the development of AIGC for vision. The database is available for download at https://www.modelscope.cn/datasets/lcysyzxdxc/AIGCQA-30K-Image.