May 2024 | Rishi Bommasani*, Kevin Klyman*, Sayash Kapoor, Shayne Longpre, Betty Xiong, Nestor Maslej, Percy Liang
The Foundation Model Transparency Index v1.1 (May 2024) evaluates the transparency of 14 foundation model developers against 100 indicators, showing a significant improvement from the previous version (v1.0) where the average score was 37 out of 100. In v1.1, the average score increased to 58, with developers disclosing new information on 16.6 indicators. The index highlights areas of sustained opacity, such as data access, labor, and downstream impact. Developers are most transparent in downstream indicators (65%) and least in upstream indicators (46%). The index shows that transparency can be improved, with some developers scoring points on 96 of the 100 indicators. The index also reveals that open developers outperform closed developers in upstream indicators but are comparable in downstream ones. The index encourages policymakers to consider interventions in areas where transparency has not improved. The index is published to enable further research and to promote transparency in the foundation model ecosystem. The findings suggest that the index contributes to improvements in transparency, and that policymakers should consider interventions in areas where transparency has not improved. The index is a tool for promoting transparency in the foundation model ecosystem.The Foundation Model Transparency Index v1.1 (May 2024) evaluates the transparency of 14 foundation model developers against 100 indicators, showing a significant improvement from the previous version (v1.0) where the average score was 37 out of 100. In v1.1, the average score increased to 58, with developers disclosing new information on 16.6 indicators. The index highlights areas of sustained opacity, such as data access, labor, and downstream impact. Developers are most transparent in downstream indicators (65%) and least in upstream indicators (46%). The index shows that transparency can be improved, with some developers scoring points on 96 of the 100 indicators. The index also reveals that open developers outperform closed developers in upstream indicators but are comparable in downstream ones. The index encourages policymakers to consider interventions in areas where transparency has not improved. The index is published to enable further research and to promote transparency in the foundation model ecosystem. The findings suggest that the index contributes to improvements in transparency, and that policymakers should consider interventions in areas where transparency has not improved. The index is a tool for promoting transparency in the foundation model ecosystem.