THE RISING COSTS OF TRAINING FRONTIER AI MODELS

THE RISING COSTS OF TRAINING FRONTIER AI MODELS

31 May 2024 | Ben Cottier, Robi Rahman, Loredana Fattorini, Nestor Maslej, David Owen
The paper examines the rising costs of training frontier AI models, which have grown significantly in recent years. It develops a detailed cost model to estimate training expenses, considering hardware, energy, cloud rental, and staff costs. The analysis reveals that the amortized cost to train the most compute-intensive models has increased by 2.4 times per year since 2016. Key expenses for models like GPT-4 and Gemini include AI accelerator chips and staff costs, each costing tens of millions of dollars. Other significant costs include server components (15-22%), cluster-level interconnect (9-13%), and energy consumption (2-6%). If the trend continues, the largest training runs will cost over a billion dollars by 2027, making only well-funded organizations capable of financing frontier AI models. The study uses three approaches to measure costs: hardware capital expenses and energy costs, cloud rental prices, and a comprehensive breakdown of hardware, energy, and R&D staff costs. The results highlight the rapid growth in AI training costs and the economic challenges that lie ahead as AI continues to scale.The paper examines the rising costs of training frontier AI models, which have grown significantly in recent years. It develops a detailed cost model to estimate training expenses, considering hardware, energy, cloud rental, and staff costs. The analysis reveals that the amortized cost to train the most compute-intensive models has increased by 2.4 times per year since 2016. Key expenses for models like GPT-4 and Gemini include AI accelerator chips and staff costs, each costing tens of millions of dollars. Other significant costs include server components (15-22%), cluster-level interconnect (9-13%), and energy consumption (2-6%). If the trend continues, the largest training runs will cost over a billion dollars by 2027, making only well-funded organizations capable of financing frontier AI models. The study uses three approaches to measure costs: hardware capital expenses and energy costs, cloud rental prices, and a comprehensive breakdown of hardware, energy, and R&D staff costs. The results highlight the rapid growth in AI training costs and the economic challenges that lie ahead as AI continues to scale.
Reach us at info@study.space
Understanding The rising costs of training frontier AI models