Trends in the dollar training cost of machine learning systems

Ben Cottier has analyzed the trends in the training cost of machine learning systems over the past decade. He has combined training compute and GPU price-performance data to estimate the cost of compute for the final training run of 124 machine learning systems published between 2009 and 2022. The results show that the cost has increased by approximately 0.5 orders of magnitude per year. This implies that the cost of training machine learning systems has gone up significantly in the past decade. The increase in cost can be attributed to advancements in technology and the increasing demand for machine learning systems.