Everyone wants to be the AI gateway - by Anh Tho Chuong

The Bill, Please

SubscribeSign in

Everyone wants to be the AI gateway<br>AI gateways look alike because they occupy the same place in the request path. They differ in the decision they control and in the business each company is building around it.

Anh Tho Chuong<br>Aug 17, 2026

Share

I kept seeing “AI gateway” attached to products that did not look like the same market.<br>Ramp launched one. Merge did too. Vercel, Cloudflare and AWS already had gateway products in market. Then two deals made the category impossible to ignore.<br>Stripe is reportedly nearing a deal to buy OpenRouter for more than $7 billion. Three months earlier, OpenRouter had raised a $113 million Series B.<br>Palo Alto Networks paid $140 million for Portkey and made it the AI gateway inside Prisma AIRS.<br>Those are not the same acquisition thesis. Stripe wants the layer that brokers model demand. Palo Alto Networks wants the layer that governs agent traffic.<br>So either this category is enormous, or the name is doing too much work.<br>My read: both.<br>“AI gateway” tells you where a product sits. It does not tell you what the product does there.<br>The position is easy to draw:<br>Application or agent → Gateway → Model, tool, API or resource<br>A company that owns that middle box can make a decision before the request runs. It also sees what happened afterward: cost, latency, usage, errors and demand.<br>Ollama helped us sharpen the boundary. It can expose an OpenAI-compatible API, but it usually executes the model itself. That makes it a runtime. A gateway sits between the caller and the runtime.<br>We first tried mapping gateway companies on two axes. It looked clean. It also made “up and to the right” look better, which was precisely the wrong message.<br>A gateway that controls more decisions also owns more latency, more failure modes and more switching cost. More surface area is not a neutral measure of quality.<br>So we dropped the ranking and grouped products by the primary decision they make.

The four jobs

01. Model routing

Which model should handle this request?<br>OpenRouter started with unified access to many models. Vercel puts routing and fallback inside a developer workflow it already owns. Ramp applies its cost-optimization instinct to model choice. Merge extends the abstraction behind its original product: one interface in front of fragmented suppliers.<br>LiteLLM and Bifrost come from the open-source proxy side.<br>Their features overlap. Their distribution does not.<br>02. Traffic control

Can this request run, and under which rules?<br>Cloudflare logs, caches and controls model traffic. Kong and Azure are extending API control planes enterprise teams already use. Envoy AI Gateway, kgateway and Traefik Hub bring cloud-native gateway infrastructure into the same path.<br>Portkey and TrueFoundry came through observability and governance.<br>The route into the customer is different even when the checkboxes start to look the same.<br>03. Agent access

Which tools can this agent use, and on whose behalf?<br>This is where the consequences change.<br>A model request returns an answer. An agent may call Salesforce, query a database or initiate a purchase. Someone has to decide which tools it may use, whose identity it carries and which credentials it receives.<br>Amazon Bedrock AgentCore Gateway, Google Agent Gateway, agentgateway and Gravitee are approaching that problem from different starting points. Some route models too. That is not the hardest part of the job.<br>04. Commercial control

What may this customer consume, and on which terms?<br>Cloudflare’s Monetization Gateway is one early expression of this job. A resource returns payment instructions, the caller pays through an HTTP 402 or x402 flow, then retries with proof.<br>That is useful when a machine buys access to a page, dataset, API or MCP tool.<br>It still does not tell you what an enterprise customer bought six months ago.<br>A customer might have an annual commitment, a monthly allowance, prepaid credits and a negotiated price for one model family. None of that lives in the model response. A successful x402 payment does not carry it either.<br>“Has this request been paid for?” and “what did this customer buy?” are different questions.<br>Why gateways are proliferating

Most companies on the map are extending an advantage they already own into a newly valuable control point.<br>Vercel has developer distribution. Cloudflare already carries the traffic. Ramp knows how to sell cost control. Merge knows how to normalize fragmented APIs. The cloud providers have infrastructure, identity and contracts their customers already trust.<br>That makes the proliferation less random than it first appears.<br>I expect the feature lists to converge faster than the businesses behind them. Routers will add policy. API gateways will add model choice. Agent gateways will front models as well as tools.<br>But a router earning margin on model spend has a different incentive from a cloud platform using a gateway to retain...