There’s an aspect I believe is occurring in the AI data center space that isn’t spoken to in the article: Landgrabbing
So many data centers are being stood up now, not because the immediate need for compute exists, but the recognition that there are bad faith players in the data center market space. Regulations largely don’t exist to prevent a new DC from being stood up or operated and there are few, if any limits on the power and water consumption these consume. That will change when the infrastructure fixed costs for water and power show up, as well as the regular rise in costs per unit of each on the variable consumption side. If these companies can get their DC approved and built before the regulations exist, or before communities block future buildout, it will be a massive asset for data center operators to prevent new competitors from entering the space.
Further, when the bubble bursts, I think we’re going to be in better shape than most bubbles. What is being built is compute infrastructure. Yes, today its being setup for AI use, but there is very little that is AI specific. Meaning, when the bubble bursts, we’ll have all this compute (then very cheap to use) as a springboard for the next industry.
A lot of the players involved very much DO remember the past few “AI Winters” and know that this tech is going to fail to impress at some point. Hence a push to ramp up with stuff they can repurpose.
But not everyone is Amazon or a good university (professor). There is going to be massive loss of employment as all the people who “know pytorch” have their demand drop to near zero. But we’ll also have an entire generation of graduates and grad students who genuinely can’t code and MAYBE can call pytorch on their data if they are particular overachievers. And we’ll have had years of “Why would I hire a junior level coder when copilot can do that for us? 20 years of experience, minimum.” And that is going to be REALLY REALLY hard to recover from.
There isn’t much in common with a GPU compute datacenter and a software or server compute datacenter besides the power and cooling. These are useless for let’s say streaming video or other content hosting.
Great for converting video (something Google does a shitload of), 3D material and fluid simulations (every engineering firm), large scientific projects (BOINC gets most of their compute from donated GPU time), and ray-tracing for movies (animated movies use a shitload of GPU compute).
They are almost certainly going to be sold at very, very low rates after the bubble pops. Lots of supply!
Think of it like economic dumping. Going to make it hard for others to compete and anybody who has demand for some of that supply is going to be in great shape!
There isn’t much in common with a GPU compute datacenter and a software or server compute datacenter besides the power and cooling.
I’m…not sure you have a good handle on how AWS, GCP, or Azure hyperscaler data centers (which are the type the article is referring to) are set up if this is your take. These services don’t sell AI GPU or compute services. Its the whole tip-to-tail of IT servers and services with AI simply being one service offering. GPUs don’t exist in as a standalone product. They’re installed in compute instances. Even the AI managed service offerings of these hyperscalers are simply compute instances chucked full of GPU with a layer of software abstraction over top of them. You don’t need to take my word for it. Its right in the sale pages for the hyperscalers:
Here’s GCP:
Compute Engine is GCP’s Virtual Machine product. GKE is GCP’s Kubernetes (compute container) product.
Here’s AWS:
EC2 is AWS’s Virtual Machine product. You can see they have many different instance types (VM types) that have GPUs available to them.
GPUs are not sold as a standalone product by these companies. They are sold in concert with the compute products and services.
The possible exception might be Oracle Cloud Infrastructure’s (OCI) Supercluster which uses RoCE to stitch together thousands of GPU cores in an AI switched fabric model. source But that is an exception, not the rule. 99% of the AI offerings from these hyperscalers are GPUs stuffed into servers that can be used for regular old non-AI compute needs.
These are useless for let’s say streaming video or other content hosting.
GPUs have been used for General Purpose (GPGPU) computing long before AI took the stage. Nvidia released CUDA, its GPGPU language 18 years ago in 2007.
There’s an aspect I believe is occurring in the AI data center space that isn’t spoken to in the article: Landgrabbing
So many data centers are being stood up now, not because the immediate need for compute exists, but the recognition that there are bad faith players in the data center market space. Regulations largely don’t exist to prevent a new DC from being stood up or operated and there are few, if any limits on the power and water consumption these consume. That will change when the infrastructure fixed costs for water and power show up, as well as the regular rise in costs per unit of each on the variable consumption side. If these companies can get their DC approved and built before the regulations exist, or before communities block future buildout, it will be a massive asset for data center operators to prevent new competitors from entering the space.
Further, when the bubble bursts, I think we’re going to be in better shape than most bubbles. What is being built is compute infrastructure. Yes, today its being setup for AI use, but there is very little that is AI specific. Meaning, when the bubble bursts, we’ll have all this compute (then very cheap to use) as a springboard for the next industry.
Yes and no.
A lot of the players involved very much DO remember the past few “AI Winters” and know that this tech is going to fail to impress at some point. Hence a push to ramp up with stuff they can repurpose.
But not everyone is Amazon or a good university (professor). There is going to be massive loss of employment as all the people who “know pytorch” have their demand drop to near zero. But we’ll also have an entire generation of graduates and grad students who genuinely can’t code and MAYBE can call pytorch on their data if they are particular overachievers. And we’ll have had years of “Why would I hire a junior level coder when copilot can do that for us? 20 years of experience, minimum.” And that is going to be REALLY REALLY hard to recover from.
Its the new “just hire someone overseas” of IT. Just another way to cut costs from hiring actual experts. Some will work, most will not.
There isn’t much in common with a GPU compute datacenter and a software or server compute datacenter besides the power and cooling. These are useless for let’s say streaming video or other content hosting.
Great for converting video (something Google does a shitload of), 3D material and fluid simulations (every engineering firm), large scientific projects (BOINC gets most of their compute from donated GPU time), and ray-tracing for movies (animated movies use a shitload of GPU compute).
We don’t need huge portions of our electrical grid dedicated to any of that.
They are almost certainly going to be sold at very, very low rates after the bubble pops. Lots of supply!
Think of it like economic dumping. Going to make it hard for others to compete and anybody who has demand for some of that supply is going to be in great shape!
None of that has anything to do with the associated electrical usage.
I’m…not sure you have a good handle on how AWS, GCP, or Azure hyperscaler data centers (which are the type the article is referring to) are set up if this is your take. These services don’t sell AI GPU or compute services. Its the whole tip-to-tail of IT servers and services with AI simply being one service offering. GPUs don’t exist in as a standalone product. They’re installed in compute instances. Even the AI managed service offerings of these hyperscalers are simply compute instances chucked full of GPU with a layer of software abstraction over top of them. You don’t need to take my word for it. Its right in the sale pages for the hyperscalers:
Here’s GCP:
Compute Engine is GCP’s Virtual Machine product. GKE is GCP’s Kubernetes (compute container) product.
Here’s AWS:
EC2 is AWS’s Virtual Machine product. You can see they have many different instance types (VM types) that have GPUs available to them.
GPUs are not sold as a standalone product by these companies. They are sold in concert with the compute products and services.
The possible exception might be Oracle Cloud Infrastructure’s (OCI) Supercluster which uses RoCE to stitch together thousands of GPU cores in an AI switched fabric model. source But that is an exception, not the rule. 99% of the AI offerings from these hyperscalers are GPUs stuffed into servers that can be used for regular old non-AI compute needs.
GPUs have been used for General Purpose (GPGPU) computing long before AI took the stage. Nvidia released CUDA, its GPGPU language 18 years ago in 2007.