Would Deep Generative Models Amplify Bias in Future Models?

Would Deep Generative Models Amplify Bias in Future Models?

4 Apr 2024 | Tianwei Chen*, Yusuke Hirota, Mayu Otani, Noa Garcia, Yuta Nakashima
This paper investigates the impact of deep generative models on potential social biases in future computer vision models. As AI-generated images become more prevalent, concerns arise about inherent biases that may accompany them, potentially leading to the dissemination of harmful content. The study explores whether a detrimental feedback loop, resulting in bias amplification, would occur if generated images were used as training data for future models. Simulations were conducted by progressively substituting original images in COCO and CC3M datasets with images generated through Stable Diffusion. The modified datasets were used to train OpenCLIP and image captioning models, which were evaluated in terms of quality and bias. The findings indicate that introducing generated images during training does not uniformly amplify bias. Instead, instances of bias mitigation across specific tasks were observed. Factors influencing these phenomena, such as artifacts in image generation or pre-existing biases in the original datasets, were explored. The results show that the behaviors of the evaluated biases are inconsistent and vary as original images are replaced with generated ones. In some cases, biases increase, while in others, they decrease. The study also highlights the potential challenges associated with the generation of facial images with Stable Diffusion, which may affect bias. The paper concludes that while images generated by Stable Diffusion exhibit bias across different demographic attributes, their use for training does not consistently amplify bias. The findings suggest that the impact of generated data may depend on the original dataset and target task, and that further research is needed to understand the complex nature of bias in AI models. The study provides recommendations for handling biased generated images in the training process of future models, contributing to the ongoing discourse on responsible and unbiased AI development.This paper investigates the impact of deep generative models on potential social biases in future computer vision models. As AI-generated images become more prevalent, concerns arise about inherent biases that may accompany them, potentially leading to the dissemination of harmful content. The study explores whether a detrimental feedback loop, resulting in bias amplification, would occur if generated images were used as training data for future models. Simulations were conducted by progressively substituting original images in COCO and CC3M datasets with images generated through Stable Diffusion. The modified datasets were used to train OpenCLIP and image captioning models, which were evaluated in terms of quality and bias. The findings indicate that introducing generated images during training does not uniformly amplify bias. Instead, instances of bias mitigation across specific tasks were observed. Factors influencing these phenomena, such as artifacts in image generation or pre-existing biases in the original datasets, were explored. The results show that the behaviors of the evaluated biases are inconsistent and vary as original images are replaced with generated ones. In some cases, biases increase, while in others, they decrease. The study also highlights the potential challenges associated with the generation of facial images with Stable Diffusion, which may affect bias. The paper concludes that while images generated by Stable Diffusion exhibit bias across different demographic attributes, their use for training does not consistently amplify bias. The findings suggest that the impact of generated data may depend on the original dataset and target task, and that further research is needed to understand the complex nature of bias in AI models. The study provides recommendations for handling biased generated images in the training process of future models, contributing to the ongoing discourse on responsible and unbiased AI development.
Reach us at info@study.space