The AI energy crisis
2025-08-04 12:13:30 +01:00 by Mark Smith
A few weeks ago I wrote about the size insanity of the new AI clusters that are being built. And I followed that up with some rather quaint context from my past experience with high performance computing in the early days of the VFX industry. Well, it seems other seasoned computer science professionals jaws are dropping too.
From a very funny, and prescient moment in a recent Changelog Podcast interview with Greg Osuri on the AI Energy Crisis Ep#652 [12:20]:
Greg: Just to give you context, this is one data center we are talking about, one company, right, so put that with the heavy competition between, what‘s his name, Mark Zuckerberg tweeted out saying, or facebook out, saying that they are going to build a data center bigger than Manhattan, we saw the tweet.
Jerod: I did not see that tweet.
Greg: Everybody is competing. Yesterday Elon Musk said they are going to, in fact, they are going to get about 5 million GPUs by 2028, I believe he said, or next 5 years, so maybe 2030. [Exhales deeply] 5 million GPUs, he said 5 million H100 ecore, that’s 5 times 1 Kilowatt, 1.2 Kilowatt per GPU, just to give you a brief idea as to how much compute or how much energy we are looking at. OH sorry 50 million. Sorry, I take the number. 50 million. These nunbers are so big.
Jerod: Not 5 million, 15 million, one five?
Greg: Five zero.
Adam: Oh 50 million, so 10x what you originally said?
Greg: Yes apologies, I mean these numbers are so large.
Greg & Adam [at the same time]: It’s hard to even fathom.
Adam: I mean 50 million GPUs!
Greg: For context, Nvidia made about 2.5 million H100s in 2024, so...
Adam: Tccchhhheeeeewww.
Greg: ...ha ha...not financial advice but, ha ha ha...but Nvidia I believe is going to...
Adam: Haha yeaaaah!
Greg: ...is going to be extremely beneficial.
Jerod: You think the stock is high now...
I asked Gemini to tabulate the 10 biggest data centers worldwide. They seem to take up to around 10 million square feet. Manhattan for reference is 633 million square feet. So that's around 60x current ginormo data centre sizes. Seems like Zuck‘s estimate was a bit ambitious, but honestly, not by that much if all these growth target numbers are to be believed. Remember Sam Altman said he wants a 100 million node cluster.
This webopedia article says that the Yotta NM1 data center in India as 30000 racks, which sounds like a heck of a lot, until you read about the Alibaba Cloud Zhangbei Data Center in China, that has 52 buildings with 50000 racks each.
Glad it‘s not just me that is OMG-ing at all this stuff. #