The AI storage pitch is a load balancing problem — Chovy's Blog

The AI storage pitch is a load balancing problem

2026-08-19, by Anthony “chovy” Ettinger.

How this was written: drafted with an AI assistant from my own notes,<br>then edited by me.

I signed up for a vendor webinar this morning, which meant giving The Register's form my name<br>and my company. F5 and Scality are presenting on 26 August about moving object data fast enough to<br>keep GPUs busy. Before deciding whether to actually show up, I went and read the technical writeup<br>the pitch is built on. The interesting part is not the storage.

The bottleneck is load balancing

A training job reads its data over S3. Clients resolve one endpoint, and whichever storage node<br>they land on gets hammered while the rest of the cluster sits idle. That is a hot spot, and it has<br>nothing to do with disks.

The fix on offer is a load balancer in front of the storage cluster, spreading connections with<br>Least Connections, or Fastest, which counts transactions currently in flight rather than<br>connections opened. There is also a Ratio mode for when half your nodes are a newer vintage than<br>the other half and you want the new ones taking more work. Global steering picks the nearest site<br>by measured latency.

None of that is AI specific. It is what people have been doing to web tiers since 2005, pointed<br>at a storage cluster instead. That is why it works.

Numbers worth stealing

Scality RING starts at three nodes and 200 TB. Objects under 60 KB get replicated, two or three<br>copies. Anything larger gets erasure coded.

EC(9,3) means nine data chunks plus three parity chunks, spread across nodes. Lose three nodes<br>and the object still reads. The cost is 33 percent overhead. EC(8,4) survives four failures at 50<br>percent. Straight replication at three copies costs 200 percent, which is why it is reserved for<br>the small stuff.

Their worked example: if 3 percent of your objects are under that 60 KB line, replicating those<br>at three copies and erasure coding the rest at EC(9,3) lands you around 41 percent total storage<br>overhead. The claimed durability is fourteen nines, which is more nines than Wikipedia's<br>availability table bothers to calculate.

I do not run petabytes. I still like knowing what the trade actually costs, because the default<br>everywhere else is three copies of everything and no thought given.

Noisy neighbors get a leaky bucket

Multi tenant storage has the obvious problem: one tenant's job eats the disk I/O and everyone<br>else's latency goes bad. The answer here is a rate limit at the edge, counting requests per client<br>address and rejecting once the count passes a threshold in a short window, with credits replenishing<br>over time. Below that, a bandwidth profile capping lower priority traffic at something like 10 Mbps,<br>and layer 4 timeouts to reap connections opened and abandoned on purpose.

Rate limit in front of the expensive thing, not inside it. Same move as putting nginx limits in<br>front of an app, only the thing being protected costs a lot more.

The part I did not expect

TLS 1.3 gets pitched on performance, one round trip to establish instead of two, and zero on<br>resume. Fine, and it adds up when a job opens a lot of connections.

The better reason is underneath. Post quantum key agreement (ML-KEM, NIST FIPS 203) needs TLS 1.3<br>to exist at all. Traffic captured today can be stored and decrypted later, possibly around 2030. For<br>a training corpus that will still be valuable in five years, harvest now and decrypt later is a<br>sharper argument than shaving a round trip.

Am I going?

Probably not. The writeup already gave me the shape, and the webinar will be the same slides with<br>logos on them.

What transfers to work my size: put a proxy in front of whatever is expensive to scale, health<br>check it with something real (a HEAD request at minimum, or an actual upload and download probe if<br>you have credentials to spare), rate limit per client, and choose your redundancy overhead on<br>purpose. Registration cost me an email address and a company name. The technical article behind it<br>is public and free.

Sources:<br>Scality RING and F5 BIG-IP on DevCentral<br>and the<br>webinar signup.

Find me: Mastodon &middot;<br>GitHub &middot;<br>email