7 Feb 2024 | Weixin Liang, Nazneen Rajani, Xinyu Yang, Ezinwanne Ozoani, Eric Wu, Yiqun Chen, Daniel Scott Smith, and James Zou
This study analyzes 32,111 AI model cards from Hugging Face, a leading platform for AI model distribution. The analysis reveals that while most models have model cards, the information provided varies significantly in completeness. Sections such as Environmental Impact, Limitations, and Evaluation are filled out less frequently than the Training section. The study also finds that discussions about data often receive as much or more attention than the models themselves. An intervention study showed that adding detailed model cards to 42 popular models increased their weekly downloads, indicating the impact of well-documented models. The findings highlight the need for more comprehensive documentation, especially regarding environmental impact and model limitations. The study underscores the importance of model cards in improving transparency, trust, and responsible AI use. It also emphasizes the role of data in AI development and the need for data-centric approaches in AI research. The results suggest that model cards can significantly influence model adoption and usage, and that improving documentation practices is crucial for ensuring fairness and accountability in AI. The study provides insights into community norms and practices around model documentation through large-scale data analysis.This study analyzes 32,111 AI model cards from Hugging Face, a leading platform for AI model distribution. The analysis reveals that while most models have model cards, the information provided varies significantly in completeness. Sections such as Environmental Impact, Limitations, and Evaluation are filled out less frequently than the Training section. The study also finds that discussions about data often receive as much or more attention than the models themselves. An intervention study showed that adding detailed model cards to 42 popular models increased their weekly downloads, indicating the impact of well-documented models. The findings highlight the need for more comprehensive documentation, especially regarding environmental impact and model limitations. The study underscores the importance of model cards in improving transparency, trust, and responsible AI use. It also emphasizes the role of data in AI development and the need for data-centric approaches in AI research. The results suggest that model cards can significantly influence model adoption and usage, and that improving documentation practices is crucial for ensuring fairness and accountability in AI. The study provides insights into community norms and practices around model documentation through large-scale data analysis.