On-Call Is Now Theatre | Boris TaneOn-Call Is Now Theatre | Boris Tane

Your on-call rotation has always been an admission of defeat. Every alert threshold, every escalation policy, every runbook is the same admission written in YAML: software cannot be trusted to run in production, so somebody must be strapped to it at all times.

But today agents write most of the code. Engineers spend their time designing, prompting, reviewing, and creating loops to continuously produce more and more software.

Building software has been transformed beyond recognition in under three years. Operating it hasn’t moved an inch. The rotation, the pager, the dashboards, the rituals: all exactly where we left them, guarding a world that no longer exists.

We’ve all lived through terrible on-call rotations

You get paged when you’re having dinner on a beautiful Saturday. You log in, ask your agent what’s up, open four dashboards, cross-reference a deploy timeline, and six “what the heck is this?” later, you conclude it’s probably not that big of a deal. You ack the alert and go back to your now-cold meal.

Nothing about that page actually needed you. It needed someone who could pull up telemetry, reason about it, and decide nothing was on fire. For the entire history of software, that someone had to be a person. Everything about how we run production is downstream of that one fact.

The core issue: everything was calibrated to human attention

Everything about on-call and observability is built around one core principle: human attention is a scarce resource.

Every alert threshold is tuned around the cost of waking up an engineer. We have always treated alert fatigue as if it was a bug in the system we could fix with better alert thresholds or SLOs. But alert fatigue is the system: a rationing mechanism for eyeballs.

Metrics exist to compress millions of datapoints into something a human can absorb in a glance. Runbooks exist to digest the knowledge of an expert in the system into a document anyone can follow. The on-call rotation itself exists to distribute suffering across a team.

When the only thing that can investigate an incident is an engineer, you design everything (thresholds, tooling, org charts, etc.) around protecting and rationing human attention.

That fundamental constraint is now obsolete.

And yet, we’re still doing it

You already let agents write the vast majority of your code. Your team probably ships dozens of PRs a day; I’ve seen teams where a single engineer ships 10+ PRs a day. We’ve industrialised the production of software with “software factories”.

But we’re still responsible for ensuring all this software runs in production, equipped with dozens of MCPs and poorly written skills. We automated the cause and kept the cure handmade.

The teams sprinting ahead with coding agents are quietly discovering that their velocity is capped not by how fast they can build, but by how fast a human can figure out what broke.

graph TD<br>A[Agents ship at machine speed] --> B[Production changes constantly]<br>B --> C[Something breaks]<br>C --> D[Page a human]<br>D --> E[Human greps, correlates, guesses]<br>E --> F[Hours of attention per incident]<br>F --> G[Velocity capped by incident response]<br>style D fill:#fee2e2,stroke:#fca5a5,color:#991b1b<br>style G fill:#fee2e2,stroke:#fca5a5,color:#991b1b<br>Every gain in build velocity converts directly into operational debt, until PagerDuty wins and the shipping stops.

Self-operating software is the next frontier for software engineering.

Self-operating software

We need software that watches itself, triages its own alerts, investigates its own incidents, fixes what it can, and escalates to a human only when it hits something genuinely novel, with the evidence already assembled.

}" data-astro-cid-26h3mjrb> Steve Faulkner (Cloudflare) on agents and on-call

document.getElementById("modal")?.classList.remove("show")" data-astro-cid-26h3mjrb> &times; Steve Faulkner (Cloudflare) on agents and on-call

Put your AI agents in the worst on-call rotation imaginable, then give them a tool to page a human. Developers stop being the first responder, and step in only when an agent genuinely cannot figure something out.

This flips the economics of what to monitor. Your thresholds are conservative because paging an engineer is expensive. If paging has near-zero marginal cost, you don’t want fewer alerts, you want dramatically more. You borderline want your agent to read every single log line and figure out all errors and unexpected paths in real-time, as requests are flowing through your systems. Monitor the p99 that crept up 3%, the queue depth that’s slightly off its weekly pattern, the error rate that’s fine but different. All the weak signals you convinced yourself are not worth monitoring usually turn into pages when it’s too late.

A friend at a lab put it extremely clearly to me recently:

“It feels like it’s going to become a non-negotiable to have harnesses programmatically access cell data, alerts,...