This paper investigates the effectiveness of data balancing in mitigating biases in contrastive language-image pretraining (CLIP). The authors propose a novel algorithm called Multi-Modal Moment Matching (M4) to reduce both representation and association biases in multimodal data. They analyze the impact of data balancing on CLIP's performance, finding that while it can improve classification accuracy, it may negatively affect retrieval performance. They also find that fine-tuning is effective in reducing representation biases but less so for association biases. The study shows that data balancing has a mixed impact on model quality, but combining it with data quality improvements and architectural enhancements can mitigate its negative effects. The authors conclude that data balancing is useful but insufficient for achieving fair downstream behavior, and recommend combining it with other intervention methods. The paper also highlights the importance of addressing biases in multimodal systems, as they can amplify societal stereotypes and lead to performance disparities. The authors emphasize the need for further research into effective bias mitigation strategies in multimodal learning.This paper investigates the effectiveness of data balancing in mitigating biases in contrastive language-image pretraining (CLIP). The authors propose a novel algorithm called Multi-Modal Moment Matching (M4) to reduce both representation and association biases in multimodal data. They analyze the impact of data balancing on CLIP's performance, finding that while it can improve classification accuracy, it may negatively affect retrieval performance. They also find that fine-tuning is effective in reducing representation biases but less so for association biases. The study shows that data balancing has a mixed impact on model quality, but combining it with data quality improvements and architectural enhancements can mitigate its negative effects. The authors conclude that data balancing is useful but insufficient for achieving fair downstream behavior, and recommend combining it with other intervention methods. The paper also highlights the importance of addressing biases in multimodal systems, as they can amplify societal stereotypes and lead to performance disparities. The authors emphasize the need for further research into effective bias mitigation strategies in multimodal learning.