The Web-Search Latency Your Agent Actually Pays - Telem AI

$20 in free credits for every new<br>user.<br>Claim yours &nearr;

&times;

&larr; All posts<br>2026-08-23 &middot; TelemAI Research

TL;DR

Cache: Every query cache we could observe expires within an hour, most<br>within 15 minutes.

p50 vs p95: A provider that looks fast at the median can be slow at the<br>tail. Serpapi's p95 is 13.6 s (3.5× its median) while ceramic's is 1.05× its median. When your<br>agent fans out and waits for every result, the slowest response sets your latency.

Snippet generation: "Snippet" can secretly mean "run an LLM". Exa's<br>LLM-written summary field costs 6.8 s and $0.017 per call versus 1.3 s and $0.007<br>for its extractive highlights, a 5× latency and 2.4× price cliff for text your downstream model<br>would happily read raw.

Ask an agent a genuinely hard question then it doesn't run a search: it runs a deep search<br>campaign. The question gets decomposed, sub-agents fan out, and every sub-task searches on its<br>own. One deep research can run on our own infrastructure issued 148 web-search calls<br>before writing a word of its answer. OpenAI tells Deep Research users up front that a single<br>answer may take 5–30 minutes[1];<br>Anthropic's research agent parallelizes its sub-agents precisely because<br>doing so cuts<br>research time by up to 90%[2]. Search is the inner loop of all of it.

Web-search latency is also one of the worst-understood numbers: the latency a benchmark<br>measures, and the latency an agent pays, are usually two different things. Here are two<br>measurements of the same search API, same request shape, same region, hours apart:

105 ms<br>query seen 90s ago

or

3,937 ms<br>query never seen before

Both numbers are real. Both are median values over 150 requests, not cherry-picked outliers.<br>The only difference between them is whether the provider had seen that query string in<br>roughly the last half hour.

That's a 37× envelope for a single provider. As a result, published<br>search-API latency numbers are so hard to reconcile with what you see in production.<br>Almost every published comparison we could find reports a single latency figure without<br>saying whether its queries were fresh. Within a 37× envelope, a number without that<br>disclosure is not interpretable.

This post is about the right-hand number: As we'll show, if you're building agents, the cache<br>essentially never helps you.

The cold price list

To compare nine providers fairly, we forced them onto one output contract (10 results, each<br>with a text snippet). We then found the cheapest request that achieves it on each API, and issued 150<br>brand-new, never-before-seen queries[5] per provider, strictly serially, with per-phase timing (the<br>numbers below are time-to-first-byte with connection setup subtracted; one network round-trip still<br>rides inside TTFB, which is why providers closer together than the geography envelope are<br>reported as ties rather than ranked). Same contract, all cold:

median TTFB (whiskers = bootstrap 95% CI)<br>p95linear axis

Three things:

The spread is 12×: 320ms to 3.9s for the same deliverable. This is<br>not a "premium vs budget" spread either; the fastest provider here is not the most expensive.

The tail is a second ranking hiding inside the first: serpapi's p95 is<br>13.6 seconds (3.5× its own median). ceramic's p95 is only 1.05× its median, which is slow but<br>metronomic. If your agent fans out to several providers and waits for all of them, the tail<br>is your latency. With a 9-way fan-out, the p50 you experience is the maximum of nine draws.<br>We measured it at 4.3s , with p95 at 13.6s.

Some rankings are not decidable from one vantage point. We measured our<br>network distance to each provider (a TLS 1.3 handshake costs one network round-trip[6]; spread 3.7–89ms). Pairs whose medians<br>differ by less than that envelope, such as you.com vs brave (16ms apart), or ceramic vs tavily (73ms),<br>are tied as far as a single measurement location can say, which is why we have not ranked them.

Q: Do these rankings survive real agent queries?

Our 150 test queries were synthetic[5].<br>The Information Retrieval literature is actually encouraging here: LLM-built test collections have been shown to<br>reproduce system rankings faithfully[7] even<br>though synthetic queries differ measurably from human ones in absolute<br>terms[8]. We also replayed the 77 queries the real<br>agent had generated (after a three-hour cool-down) through the same instrument, paired across<br>all nine providers, with 693 calls and zero failures. We also included a batch of agent and<br>synthetic queries fired within the same minutes, so time-of-day can't tip the scale.<br>Rank agreement with the synthetic set: Kendall τ = 0.71[9]<br>by our preregistered rule, 0.90 after also excluding the provider whose latency<br>appears server-state-driven rather than query-driven. And side by side, the two kinds of query<br>are simply indistinguishable:

synthetic queries<br>real agent queriessame minutes, alternating

Every interval contains, or in brave's case grazes,...