You're not buying intelligence. You're buying amnesia.<br>Execution layer<br>Regions<br>Resources<br>Pricing<br>Docs

Sign InGet started<br>Get started

Share

Infrastructure

August 5, 2026

It's the amnesia, stupid. Voice AI's missing cache layer.<br>Every expensive computation in computing history got a cache in front of it. Voice AI is the exception. Why the cascade's seams are the product.

Luke Miller<br>Co-founder

In voice, latency is the product.

Every other interface gets a cheap way to hide its delays. A webpage renders progressively, a chatbot shows a typing indicator, an app puts up a spinner and you forgive it for ten seconds, free of charge. A phone call gets a filler phrase like "let me just pull that up for you" and that's the whole toolkit.

It buys two seconds, maybe three, before it curdles into stalling. Humans run turn-taking on a clock measured in milliseconds; a pause of two hundred feels attentive, a pause of two thousand feels broken, and one filler phrase is all the cushioning the medium allows. The caller doesn't think "high P99." They think nobody's there, and then they hang up.

The entire user experience of a voice agent comes down to who controls the silence.

Right now the answer, for almost every developer, is: NOT you. Every turn of every conversation round-trips a frontier model in someone else's region, behind someone else's queue, and your product's feel on any given afternoon depends on that company's load that afternoon. You don't have a latency problem. You have a latency landlord.

Now look at where the wait and the bill actually come from. In a cascade turn, two meters dominate: the language model generating the response, and the synthesis speaking it.

And audit what they're generating. The LLM is produced, token by token, at full compute, with the entire conversation re-fed as input every single turn - confirmations, readbacks, templated responses: sentences that are 90% the same sentence, call after call.

Then the TTS renders that mostly-identical speech fresh, as if it had never been spoken before.

The industry is quietly admitting the point.

Yes, the savvy teams already hand-cache their greetings and filler phrases, and good for them. But a cached greeting is where the reuse ends, because the moment a sentence has a variable in it, exact-match caching is useless.

"Your appointment is Tuesday the 14th at 3:15" gets said, in shape, millions of times a day, one slot changing, everything else identical, and every single one is generated fresh and synthesized fresh, twice through the meter. That's most of what an agent actually says.

And around those two meters, the same expensive loop is also picking tools and validating inputs - decisions a lookup table could make.

A CPU from 2009 handles the structural work without noticing. We put all of it on the scarcest silicon on earth, in the one position where every millisecond is audible.

It's amnesia. Very expensive amnesia.

To be precise about the patient: I'm talking about voice agents on the cascade architecture. You have an orchestrator coordinating speech-to-text, a language model, and text-to-speech, which is how nearly every production voice agent runs today. There are other architectures, and choosing between them is a discussion for another day.

The moment this clicked for us wasn't a latency graph. It was watching our customers fight non-deterministic models for reliability while all of them were wrestling with the same two problems:

Getting model calls to behave dependably across the stack,

Getting the same question to produce the same answer twice.

Most of the agent is not a model.

Somewhere in the middle of that it landed: the industry's reflex is to throw a model at every problem, when for most of the work there are tools that are faster, cheaper, more reliable and already extraordinarily distributed, in a way high-end GPUs very much are not.

We over-attribute agents to models the way we once over-attributed websites to servers — the behavior looks like intelligence, so we assume it's all inference.

It isn't.

Most of it is structure, and structure runs anywhere. A voice agent shouldn't outsource all its thinking to a non-deterministic LLM. We just need to pull in some older, less sexy tooling.

This argument is older than the web browser.

I didn't come up with the underlying law and neither did anyone in voice. Computing has now had to relearn it four times.

In 1993 the web was two years old and arguing about whether HTML should have an image tag. In the very thread where Marc Andreessen proposed the tag, he was already defending what he called the single-hop principle i.e. a page should arrive in one round trip, against proposals that would piece pages together on the fly. The document web's first architecture fight was a latency fight.

In 1996, as dial-up brought the internet to consumers, Stuart Cheshire wrote the rant that named the law: "It's the latency, stupid." Bandwidth improves forever, latency has a...