The Foundation Model Transparency Index v1.1 May 2024

The Foundation Model Transparency Index v1.1 May 2024

May 2024 | Rishi Bommasani*, Kevin Klyman*, Sayash Kapoor, Shayne Longpre, Betty Xiong, Nestor Maslej, Percy Liang
The Foundation Model Transparency Index (FMTI) v1.1, launched in May 2024, assesses the transparency of leading foundation model developers, following a similar study in October 2023 (v1.0). The v1.1 study evaluates 14 developers against 100 transparency indicators, with developers submitting their own reports, potentially including previously undisclosed information. The average score improved from 37 out of 100 in v1.0 to 58 in v1.1, a 21-point increase. Developers disclosed information on an average of 16.6 new indicators, with significant improvements in upstream resources, model capabilities, and downstream use. However, areas like data access, impact, and trustworthiness remain highly opaque. The study highlights the potential for further transparency improvements and suggests that policymakers should consider interventions in these areas. The findings indicate that while there is significant room for improvement, the FMTI has contributed to increased transparency and that developers are willing to disclose more information when given the opportunity.The Foundation Model Transparency Index (FMTI) v1.1, launched in May 2024, assesses the transparency of leading foundation model developers, following a similar study in October 2023 (v1.0). The v1.1 study evaluates 14 developers against 100 transparency indicators, with developers submitting their own reports, potentially including previously undisclosed information. The average score improved from 37 out of 100 in v1.0 to 58 in v1.1, a 21-point increase. Developers disclosed information on an average of 16.6 new indicators, with significant improvements in upstream resources, model capabilities, and downstream use. However, areas like data access, impact, and trustworthiness remain highly opaque. The study highlights the potential for further transparency improvements and suggests that policymakers should consider interventions in these areas. The findings indicate that while there is significant room for improvement, the FMTI has contributed to increased transparency and that developers are willing to disclose more information when given the opportunity.
Reach us at info@study.space
Understanding The 2024 Foundation Model Transparency Index