Can the Cloud Be Disrupted With AI?

For the sake of argument, let’s assume cost of software will go to zero because of AI.1 Could that change cloud economics, such that new players can chip away at the AWS/GCP/Azure moat?

First off: what do “cloud providers” do?

Cloud providers build/rent datacenters, setup the networking, and maintain it all. They then “virtualize” the hardware and sell pieces of it (CPU, Disk, etc.) to customers. CTOs are happy to pay cloud providers a premium to deal with the pain of handling hardware. In return, their software engineers get servers “on-demand” through an API call.

Most customers need to run commonly used open source software (Postgres, Elastic, Redis, etc.). As a convenience, cloud providers do this for them, package it as a “product” like RDS, and sell the service at a premium.

If the cost of software is zero, then 2) is no longer a differentiator (why pay RDS prices when you can save the margin using EC2 directly?). Even more important, battle-hardened hardware virtualization - and all the software to operate that at scale - would also be available to everyone.

This will drive down costs, but how do we untangle 1) from the grips of cloud providers?

Multi-Cloud Isn’t Easy

Even though cloud providers are conceptually fungible, in practice they are not. For most companies, it takes a great deal of software infrastructure, know-how, and process to swap out an AWS EC2 instance for a GCE instance, even if you’re a Kubernetes Pro. And this even more true if we consider lesser-known providers like Linode, Hetzner, Digital Ocean, etc.

Even though server hardware is a commodity that’s sold as virtualized slices, there isn’t a “market” for compute/storage where any instance is seamlessly interchangeable with another.

The main inefficiency is that a “cloud provider” tightly couples building/maintaining hardware with software services. That may be optimal for their margins, but in reality hardware and software require their own specializations and an efficient economy should reflect that. This is all the while product companies foot the bill and handcuff themselves to vendor lock-in.2

How do we truly decouple hardware from software, from an economic perspective? The challenge may be making hardware “fungible” enough. Here I’m interested in how other industries ended up “creating markets”.

Market Infrastructure for Server Hardware

Let’s imagine a world where we had a two-sided market of these players:

Hardware companies that build/rent/maintain datacenters, PoPs, etc. With AI, they too can virtualize and sell their hardware without needing to be software experts.

Product companies that need to deploy their software. They no longer need special products like RDS, Elasticache, etc., since AI can handle open source versions for them.

What’s needed to allow bids, asks, transactions, clearing etc. between these two groups? Yep, you guessed it - we need similar entities that uphold financial markets, like a clearing house, exchange, etc. This gives us market-rate pricing for most cases, and when businesses want to “reserve instances”, they can of course buy futures on virtualized instances. Plus, if a business has bought too many, they can offload the surplus back to the market!3

The most useful feature of a true “server market” is that it frees up agents to buy/sell capacity as needed without all the hassle that exists today. I suspect the demand here is much higher than we’d expect. If so, it could convince more suppliers to come onboard to make compute even more available.

… now back to reality

Conveniently, I left out two really important technical blockers: networking and security.4

If I get a “fungible instance” from AWS, Linode, and Hetzner, how do I actually wire them up for a single app with no disruption? Do they somehow have my DNS records? Are they part of the same private network behind a NAT? What’s the server-to-server latency? How do I guarantee fine-grained access? How do I know it’s ok to get an instance from another cloud provider? None of this is obvious.

And the elephant in the room… why wouldn’t cloud providers fight this tooth and nail? This would blow away their margin!

I don’t have great answers to these, and even if I did, the road to make this real is long. I suspect we’ll need some catalyst to spur folks into action. For example, if/when the AI bubble does pop, there will be an incredibly large oversupply of compute - idle data centers, listening to the grass grow. Given the financial, political, and social investment in these data centers, there’ll be an overwhelming incentive to keep them generating revenue and keep people employed. Perhaps at that moment, market makers will come in to distribute compute to every day agents.

Footnotes

It won’t. However, it may someday get cheap enough to disrupt cloud companies. ↩

Vendor lock-in is slightly more manageable with agents. But no migration is without risk (especially opportunity cost),...