It is common knowledge that machine learning a Very of energy. All those AI models that power email summaries, registered chatbots, and Homer Simpson’s nu-metal song videos are racking up a hefty server bill measured in megawatts per hour. But it seems no one — not even the companies behind the tech — can say exactly what the price is.
There are estimates, but experts say these figures are sketchy and partial, offering only a glimpse of AI’s total energy consumption. This is because machine learning models are incredibly variable, able to be configured in ways that dramatically change their power consumption. Additionally, the organizations best placed to foot the bill—companies like Meta, Microsoft, and OpenAI—just aren’t sharing relevant information. (Judy Priest, CTO for cloud operations and innovation at Microsoft, said in an email that the company is currently “investing in developing mechanisms to reduce the energy use and carbon footprint of AI while large systems is working on ways to make it more efficient, both training and application.” OpenAI and META did not respond to requests for comment.)
One important factor we can identify is the difference between training a model for the first time and rolling it out to users. Training, in particular, is energy intensive, consuming far more electricity than traditional data center activities. Training a large language model like GPT-3, for example, is estimated to use just under 1,300 megawatt hours (MWh) of electricity. About as much electricity as 130 American homes use annually. To put this into context, one hour of streaming Netflix requires about 0.8 kWh (0.0008 MWh) of electricity. That means you’d have to watch 1,625,000 hours to use the same amount of power it takes to train GPT-3.
But it’s hard to say how such a figure applies to current state-of-the-art systems. Energy consumption can be high, as AI models have been steadily increasing in size over the years and larger models require more energy. On the other hand, companies can use some proven methods to make these systems more energy efficient – which will reduce the upward trend in energy costs.
Sascha Lucioni, a researcher at the French-American AI firm Hugging Face, says the challenge with the latest estimates is that companies have become more secretive as AI has become profitable. Go back just a few years and firms like OpenAI would publish details of their training systems – on what hardware and for how long. But the same information doesn’t exist for newer models, such as ChatGPT and GPT-4, Luccioni says.
“With ChatGPT we don’t know how big it is, we don’t know how many parameters are in the underlying model, we don’t know where it’s running … it could be three raccoons in a trench coat because you just don’t know. . Know what’s under the hood.”
“It could be three raccoons in a trench coat because you don’t know what’s under the hood.”
Luccioni, who has written several papers examining AI energy use, suggests that this secrecy is partly due to competition between companies but is also an attempt to deflect criticism. Energy consumption statistics for AI – especially its most wasteful use cases – naturally invite comparison to cryptocurrency wastefulness. “There’s a growing awareness that not everything comes for free,” she says.
Model training is only part of the picture. After a system is built, it is rolled out to users who use it to generate output, a process known as “distribution.” Last December, Luccioni and colleagues at Hugging Face and Carnegie Mellon University published a paper (currently awaiting peer review) that was the first estimate of the energy consumption of various AI models.
Lucioni and his colleagues ran tests on 88 different models covering a variety of use cases, from answering questions to identifying objects and creating images. In each case, they ran the task 1,000 times and estimated the energy cost. Most of the tasks they tested used small amounts of energy, such as 0.002 kWh for classifying writing samples and 0.047 kWh for generating text. If we use our hours of Netflix streaming as a comparison, these are equivalent to the energy spent watching nine seconds or 3.5 minutes, respectively. (Remember: This is the cost of performing each task 1,000 times.) The figures were particularly large for the image generation models, which consumed an average of 2.907 kWh per 1,000 iterations. As the paper notes, the average smartphone uses 0.012 kWh to charge – so can generate a photo using AI. Use about the same amount of energy to charge your smartphone.
The emphasis, though, is on “can” because these figures are not necessarily generalizable to all use cases. Lucioni and his colleagues tested ten different systems, from small models producing tiny 64 x 64-pixel images to large models producing 4K images, and the results yielded a wide spread of values. The researchers also standardized the hardware used to better compare different AI models. This may not necessarily reflect real-world deployments, where software and hardware are often optimized for energy efficiency.
“It’s certainly not representative of everyone’s use case, but now at least we have some numbers,” Lucioni says. “I wanted to plant a flag in the ground saying, ‘Let’s start here’.”
“The productivity AI revolution comes with planetary costs that are completely unknown to us.”
The study provides useful relative data, then, though not absolute figures. It shows, for example, that AI models require more power to generate output than to classify input. It also shows that anything that contains imagery has more energy than text. Lucioni says that while the nature of this data may be disappointing, it tells a story in itself. “The productivity AI revolution comes with planetary costs that are completely unknown to us and to me the proliferation is particularly telling,” she says. “tl;dr we just don’t know.”
So trying to minimize the energy cost of producing a single Balenciaga poop is difficult because of the quagmire of variables. But if we want to better understand planetary costs, there are other steps. What if, instead of focusing on model estimation, we zoomed out?
That’s the approach of Alex de Vries, a PhD candidate at VU Amsterdam who cut his teeth calculating bitcoin’s energy costs for his blog. Digiconomist, and who have used Nvidia GPUs – the gold standard of AI hardware – to estimate the sector’s global energy use. As explained in the commentary published in de Vries. Joule Last year, Nvidia had about 95 percent of sales in the AI market. The company also releases energy specs for its hardware and sales projections.
Combining this data, de Vries calculates that the AI sector could use 85 to 134 terawatt hours per year by 2027. This is equivalent to the annual energy demand of De Vries’ home country, the Netherlands.
“You’re talking about AI power consumption that will potentially be half a percent of global power consumption by 2027,” explains de Vries. the edge. “I think that’s a pretty significant number.”
A recent report by the International Energy Agency has made similar projections, suggesting that the demand for AI and cryptocurrency will significantly increase power consumption by data centers in the near future. The agency says current data center energy use is about 460 terawatt hours in 2022 and could grow to between 620 and 1,050 TWh in 2026 – equivalent to the energy needs of Sweden or Germany, respectively.
But de Vries says it’s important to put these figures in context. He notes that between 2010 and 2018, data center energy use has remained fairly stable, accounting for about 1 to 2 percent of global consumption. (And when we say “data centers” here, we mean everything that makes up the “Internet”: from the internal servers of corporations to all the apps you can’t use offline on your smartphone. De Vries say, but the hardware became more efficient, thus offsetting this increase.
His fear is that things could be completely different for AI because companies tend to just throw bigger models and more data at anything they do. “It’s a really deadly dynamic for performance,” says de Vries, “because it creates a natural incentive for people to just keep adding more and more computational resources, and as the models or hardware become more efficient. Go, people will make these models even bigger than before.”
The question of whether the increase in efficiency will meet the increased demand and usage is impossible to answer. Like Luccioni, de Vries laments the lack of available data but says the world cannot simply ignore the situation. “It’s a bit of a hack to work out which direction it’s going and it’s definitely not an absolute number,” he says. “But that’s enough ground to give a little warning.”
Some companies involved in AI claim that the technology itself can help with these problems. Speaking for Microsoft, Priest said AI “will be a powerful tool to drive sustainability solutions,” and emphasized that Microsoft is “sustainably carbon negative, water positive and zero waste by 2030.” working to reach the goals of
But one company’s goals can never encompass the needs of an entire industry. Other methods may be required.
Lucioni says she would like to see companies introduce Energy Star ratings for AI models, allowing consumers to compare energy efficiency in the same way they can for appliances. For de Vries, our approach should be more fundamental: Do we even need to use AI for a specific task? “Because considering all the limitations of AI, it’s probably not going to be the right solution in a lot of places, and we’re going to be wasting a lot of time and resources trying to figure it out the hard way,” he says.