GitHub Outages Show the Limits of Reactive Scaling — Rahmi Pruitt

On August 17, GitHub had a nearly eight-hour outage that is worth studying if you care about<br>agent infrastructure. The interesting part was not simply that GitHub had "too much traffic."<br>The interesting part was that a hidden concurrency limit was crossed, autoscaling was watching<br>the wrong layer, load balancers saturated, and retry behavior amplified demand during recovery.

GitHub says the outage lasted 7 hours and 47 minutes and disrupted GitHub.com, authentication,<br>GitHub Actions, APIs, pull requests, issues, and Copilot. The Register's writeup says the<br>immediate cause involved network saturation on load balancers after an Istio sidecar reached a<br>concurrency limit, with recovery worsened by a latent retry bug in Visual Studio Code.12

That is the hard part about reactive scaling. It works when the bottleneck is obvious and<br>demand is relatively well-behaved. It struggles when the system crosses a hidden limit and<br>clients react by creating even more demand.

I would bet GitHub already has retries, jitter, circuit breakers, load balancing, autoscaling,<br>and every other modern reliability pattern people tell you to use. This is GitHub, not<br>somebody's weekend Django app running on a potato. The lesson is not that those tools are bad.<br>The lesson is that they may not be enough when demand itself becomes reactive.

For people who care about benchmarking, this is familiar. A machine does not fail cleanly<br>after its stable limit. Once it gets past a certain concurrency point, extra requests do not<br>merely fail. They consume sockets, CPU, memory, database connections, queue slots, logs, and<br>worker time. Past the limit, failed work can start reducing the throughput of successful work.

partial failure &rarr; retries &rarr; more traffic &rarr; more saturation &rarr; more errors &rarr; more retries

What is interesting to me is that, at lower layers of the internet, we already understand<br>pacing. We do not just flood networks with packets forever and hope for the best. We listen<br>for congestion signals. We speed up. We slow down.

But at Layer 7, a load balancer can watch a fleet get half-taken out by capacity pressure and<br>continue routing traffic until the fleet is fully saturated. The standard response is to add<br>more capacity and push the rest back onto the client: "Use better retries. Add jitter. Add<br>exponential backoff. Add a circuit breaker."

Those tools are good. They have carried the industry for a long time. But agents threaten the<br>assumptions underneath them.

To be clear, when I say agent-native infrastructure, I do not mean infrastructure controlled by<br>agents. I mean infrastructure built with agents in mind: bursty traffic, recursive workflows,<br>parallel tool calls, retries, long-running jobs, shared downstream limits, and workloads that<br>can move much faster than human clicking speed.

For most of the web, one human action roughly maps to one request, or at least a predictable<br>handful of requests. Agents change that. One user goal can fan out into many tool calls, API<br>calls, browser sessions, database queries, CI jobs, model calls, retries, and follow-up tasks.

That is why agent-native infrastructure is becoming a real category and not just marketing<br>soup. Other people are starting to point at the same shift: a16z has written about agent<br>workloads creating coordination problems around routing, locking, state management, and policy<br>enforcement across massive parallel execution.3 I'd add pacing and<br>fairness to that list — deciding how fast work should be allowed to flow, and making sure one<br>workload's burst doesn't starve everyone else's.

GitHub is also in a unique position. It has a massive free product that agents, editors, CI<br>systems, and developer tools can all push against at their own pace. Unlike platforms that can<br>simply price every call or hide behind strict quotas, GitHub has to provide a globally reliable<br>developer platform that many machines treat as shared infrastructure.

New architectures like Cursor's Origin are interesting here too. Origin is built on Continuity,<br>Cursor's storage layer, which rethinks Git hosting around a write-ahead log in S3-compatible<br>object storage, with replicas catching up from that source of truth. That may improve the<br>origin side of Git hosting.4

But even better origins do not remove the downstream problem. If a push, webhook, CI trigger,<br>deployment pipeline, or agent workflow fans out to systems like Depot, Vercel, Buildkite, GitHub<br>Actions, caches, databases, or internal APIs, those consumers still have to absorb the burst.<br>Storage architecture can move a bottleneck. It does not magically pace every downstream system<br>that reacts to the event.

Reactive systems scale supply. Retry storms amplify demand.<br>When demand reacts faster than supply can scale, the missing layer is not just load balancing.<br>It is demand shaping.

I built Aquifer around that missing control loop.

Aquifer is a self-hosted...