The paper "The Compute Divide in Machine Learning: A Threat to Academic Contribution and Scrutiny?" by Tamay Besiroglu, Sage Andrus Bergerson, Amelia Michael, Lennart Heim, Xueyun Luo, and Neil Thompson explores the significant differences in computing resources between industrial and academic AI labs. The authors provide a data-driven survey to understand how the compute divide affects machine learning research, particularly in the context of large-scale models and foundation models. They find that the compute divide has led to a reduced representation of academic-only research teams in compute-intensive research topics, especially in the development of large self-supervised models. This shift has resulted in a noticeable increase in academic research focusing on open-source, pre-trained models developed by industry. The paper highlights several implications of the compute divide, including reduced scrutiny and evaluation of influential models, and the potential for reduced diffusion of machine learning systems. It recommends approaches to address these challenges, such as expanding academic insights through nationally-sponsored computing infrastructure and open science initiatives. Structured access programs and third-party auditing are also suggested to ensure that industry models receive necessary scrutiny and evaluation. Key findings include: - The proportion of large-scale machine learning models created by academic labs has decreased from 65% in the early 2010s to 10% in the early 2020s. - Industry-only research teams have dominated the training of large-scale models, reaching 81% in 2022. - The compute required to train machine learning models has doubled every six months since the early 2010s. - Industry-affiliated research groups report using hardware five times more powerful than non-industry groups. - The compute divide is less pronounced in Chinese AI development due to government-sponsored support. - Open-source ML models like BERT have become ubiquitous baselines in NLP experiments, with a significant increase in research papers studying them. The paper concludes with policy recommendations to promote and support research that may be under-provided by industry labs, emphasizing the importance of responsible compute provision, open science initiatives, and structured access programs.The paper "The Compute Divide in Machine Learning: A Threat to Academic Contribution and Scrutiny?" by Tamay Besiroglu, Sage Andrus Bergerson, Amelia Michael, Lennart Heim, Xueyun Luo, and Neil Thompson explores the significant differences in computing resources between industrial and academic AI labs. The authors provide a data-driven survey to understand how the compute divide affects machine learning research, particularly in the context of large-scale models and foundation models. They find that the compute divide has led to a reduced representation of academic-only research teams in compute-intensive research topics, especially in the development of large self-supervised models. This shift has resulted in a noticeable increase in academic research focusing on open-source, pre-trained models developed by industry. The paper highlights several implications of the compute divide, including reduced scrutiny and evaluation of influential models, and the potential for reduced diffusion of machine learning systems. It recommends approaches to address these challenges, such as expanding academic insights through nationally-sponsored computing infrastructure and open science initiatives. Structured access programs and third-party auditing are also suggested to ensure that industry models receive necessary scrutiny and evaluation. Key findings include: - The proportion of large-scale machine learning models created by academic labs has decreased from 65% in the early 2010s to 10% in the early 2020s. - Industry-only research teams have dominated the training of large-scale models, reaching 81% in 2022. - The compute required to train machine learning models has doubled every six months since the early 2010s. - Industry-affiliated research groups report using hardware five times more powerful than non-industry groups. - The compute divide is less pronounced in Chinese AI development due to government-sponsored support. - Open-source ML models like BERT have become ubiquitous baselines in NLP experiments, with a significant increase in research papers studying them. The paper concludes with policy recommendations to promote and support research that may be under-provided by industry labs, emphasizing the importance of responsible compute provision, open science initiatives, and structured access programs.