markjgsmith

50 million nodes is insane

There is a pretty eye popping table in this Tom‘s Hardware article that shows the Nvidia enterprise GPU roadmap. It crazily parabolic.

From a not too shabby 3 TB/s memory bandwidth to a truly ridiculous 32 TB/s, in only 3 years. For context, consumer memory bandwidth for DDR5 is still around 35 GB/s! Some context from a different angle, a fully rendered full feature 4K resolution Hollywood movie at a VFX studio that artists work on is around 10-14 TB in size. Now remember what they have been saying recently, that these generative AIs will eventually be able to generate entire movies in an instant, and you start to realise the significance.

Oh yeah and Elon is saying he will have quite a big collection of these. Last week Sam Altman said they were on track to having 2 million of them and now Elon says he'll have 50 milion.

And then there is this: "50 million of these processors will consume 35 gigawatts (GW), which is equal to the typical power generated by 35 nuclear power plants". When I started out in the VFX industry in the mid 2000s, our render farm had around 250 nodes, that was pretty big at the time.

50 million nodes. This number is just so big. I am having trouble believing this is possible. #

The ruliad and possibilities space

This concept popped up a few times in my mind in the past few weeks for some reason. I can never remember the name of it. I finally had some time today to look it up. The ruliad describes the possibilities space of all possible computations, all possible inputs over an infinite number of steps. In a way, it‘s possibly the most ridiculous idea ever. And yet, there is a lot you can start to discover when you attempt to formally describe, well, everything.

I didn‘t have the time to read it all, it‘s a very long article, but once you get the general concept, the sections on experiencing the ruliad, living in the rulial space, what is is beyond the ruliad?, and especially communicating across rulial space and alien views of the ruliad, are incredibly fascinating imo. It gets very scientific in places, but just skip over the crazy diagrams. I have listened to Wolfram talking about it himself on some podcasts, and that‘s a much easier way to absorb the ideas than text to be honest.

For some reason setting up automated Typescript typechecking causes me to think about this kind of stuff.

Anyway this post is mostly so I can easily find the idea again, but also it might be interesting to others. #