OpenRouter vs Vercel vs LLMGateway Performance | LLM Gateway

Search models by provider, name, ID, or alias…⌘K<br>DevPass<br>Chat<br>Pricing<br>Docs<br>Models<br>ProductsAI Gateway<br>DevPass<br>Chat Playground<br>Observability

ResourcesEnterprise<br>Blog<br>Changelog<br>Integrations<br>Reliability<br>Guardrails<br>Providers<br>Apps<br>Models<br>Model Timeline<br>Compare<br>Token Cost Calculator<br>Referral Program

AIMCP Server<br>Agents<br>AI SDK Provider<br>Agent Skills<br>Templates<br>Guides

1.4k

Log Insvg]:px-3 bg-zinc-900 dark:bg-white text-white dark:text-black hover:bg-zinc-700 dark:hover:bg-zinc-200 font-medium w-full md:w-fit" href="/signup">Get Started

Back to blogCopy Markdown

Every AI gateway adds a hop between your app and the model. The question that matters is what that hop costs at the moment your user is staring at a blank chat window: the time to first token. Most gateway latency debates skip the measurement and argue architecture — so we measured it.<br>We ran an open-source TTFT benchmark against LLM Gateway and OpenRouter, interleaved, from the same machine, on the same model. The medians over 75 runs each: LLM Gateway got the first content token in 906ms on a cold connection and 814ms on a warm one. OpenRouter took 1392ms and 1232ms. That is roughly 35% faster cold and 34% faster warm, with zero errors across the 300 measured runs — 450 HTTP requests in total, counting the throwaway warm-up call that precedes each warm measurement, every one of which returned HTTP 200. The raw per-run data is published in full.<br>How we measured AI gateway performance<br>We used ai-gateways-benchmark, an open-source script by Ronny Badilla that recently made the rounds comparing Vercel AI Gateway, OpenRouter, and Cloudflare AI Gateway. It is Python stdlib only — raw sockets, no HTTP library — and it times every phase of a streaming request separately: DNS, TCP connect, TLS handshake, TTFB (request sent to first response byte), and TTFT (request sent to first content token in the SSE stream).<br>That separation is the point. "Gateway latency" claims usually conflate connection cost, edge proximity, and actual routing overhead. This tool splits them apart.<br>Our setup, on July 22, 2026:<br>Same model on both gateways : claude-haiku-4.5, streaming, max_tokens: 16, identical prompt<br>75 cold + 75 warm runs per gateway , interleaved round-robin so time-of-day drift hits both equally<br>Cold = a fresh connection paying full DNS + TCP + TLS with a new TLS context (no session resumption)<br>Warm = a second request on an already-open socket — the connection-pool case your production traffic mostly lives in<br>One residential vantage point, zero errors on either gateway across all 450 HTTP requests (including warm-ups)<br>Results: LLM Gateway vs OpenRouter<br>Medians of n=75 per cell. Cold TTFT is end-to-end: DNS + TCP + TLS + time to first content token — what a short-lived process pays.<br>Medians TTFB (cold) TTFT (cold) TTFT (warm) LLM Gateway 201ms 906ms 814ms OpenRouter (direct) 1369ms 1392ms 1232ms The spread, as median with p10–p90 range:<br>Metric LLM Gateway OpenRouter Cold TTFB 201 (194–240) 1369 (979–1643) Cold e2e TTFT 906 (803–1380) 1392 (1002–1675) Warm TTFB 176 (171–193) 1229 (924–1500) Warm TTFT 814 (673–1379) 1232 (924–1501) Two things stand out beyond the medians. LLM Gateway's p90 cold TTFT (1380ms) came in below OpenRouter's median — the slow tail of one distribution beat the middle of the other. And LLM Gateway's warm TTFB barely moves: 171–193ms across the p10–p90 range, which is the stability you want under a production connection pool.<br>Where the time goes<br>The phase breakdown shows the overhead is not in the connection:<br>Gateway DNS TCP TLS TTFB TTFT (request) LLM Gateway 3.2 20.9 40.2 200.9 829.5 OpenRouter 3.4 7.2 12.7 1369.4 1369.8 OpenRouter's edge actually wins the handshake — 13ms TLS vs our 40ms. It loses everything after the request is sent.<br>One honest caveat on the TTFB column: it flatters LLM Gateway for an architectural reason. Our gateway starts the response stream in about 200ms, before the first upstream token arrives. OpenRouter holds its first byte until the first token is ready, so its TTFB equals its TTFT on every single run. TTFB tells you who streams headers early; TTFT is the number your users feel, and it is the honest headline of this post.<br>A second caveat: each gateway ran its default routing for this model. OpenRouter picks its upstream per request (a request we inspected was served via Amazon Bedrock), while our runs were pinned to Anthropic's API. Both are what you get out of the box, but they are different upstreams.<br>How Vercel AI Gateway compares<br>We did not benchmark Vercel ourselves. The benchmark's author published his own run of the same script — from his vantage point, on a different date — comparing Vercel AI Gateway, OpenRouter, and Cloudflare AI Gateway proxying OpenRouter:<br>His run (medians of n=5) TTFB TTFT (cold) TTFT (warm) Vercel AI Gateway 785ms 1099ms 822ms OpenRouter (direct) 1100ms 1123ms 986ms Cloudflare → OpenRouter 1300ms 1420ms 1279ms These numbers are...