From Chat Completions to an Agents API – Notes by Dennis Yurkevich

From Chat Completions to an Agents API<br>AIagentsMCPservice-as-software<br>This Thursday I was at an AI event hosted by Crusoe and heard a talk by Varun Randery from poolside. He used a term that was new to me: the "Agent API". His point was that we are moving away from chat completions, where you send text and get text back, toward a different model, where you send a unit of work and get the finished result back.

As someone who is spending a large chunk of his time thinking about and building agents, I found this a rather depressing thought. If the shift happens, what is left to build for those of us who do not work at a frontier lab? The labs are clearly not content to be (only) an intelligence API. They want to absorb the infrastructure around agents and workflows too. The obvious reason is revenue. The services and salary budget is far larger than the software budget, and they want access to it. A second reason, and one I believe is the true reason, is training data: every user-accepted unit of work is a signal about what good work looks like.

The Agent API

I messaged Varun to check I had understood him, and he pointed me to OpenAI's Secure MCP Tunnel as evidence. The tunnel lets a hosted OpenAI product reach into your private systems through an outbound-only connection, so you never expose your MCP server to the internet. You run a small client inside your network, it polls OpenAI for queued work, forwards each request to your private server, and returns the response through the same path. ChatGPT or Codex can now read the data behind a COBOL service written twenty years ago. Intelligence is not being pulled into the organisation, the organisation's work is being pulled into the API.

The slide that sent me down this rabbit hole.

Service-as-Software

After noodling on this for some time I realised that this concept is not new. It has been discussed for a while under the name Service-as-Software, a framing Foundation Capital's Joanne Chen and Jaya Gupta wrote about in early 2024. The framing is that SaaS changed how software was delivered, and Service-as-Software changes how labor is delivered. People are excited because it moves the conversation from software budgets to salary budgets. Foundation Capital put numbers on it: Salesforce makes around $35B a year, a whisker of the roughly $1.1 trillion spent globally on sales and marketing salaries. In another post, they size the full services opportunity at $4.6 trillion against a $200B SaaS market.

The land grab

It feels as though there is a value-chain land grab going on here. The incumbent SaaS view is that SaaS evolves into an integration and orchestration layer, with agents sitting on top of existing platforms while those platforms keep the data and workflows. The idea of the Agents API points the other way. The labs return the finished work and route around the SaaS app entirely, capturing the labor budget directly. Technology like the MCP tunnel is what makes that disintermediation technically possible.

What's left

So back to my worry. If the agentic loop moves out of the organisation, and the organisation becomes mostly a system of record that the labs dip into when they need to perform work, what is left?

A few options come to mind:

Specification . Is the actual unit of work well specified?

Context . The labs can reach the raw records, but are raw records enough?

Grading . Completed work has to be evaluated and graded to confirm it is

correct.

The trouble is that the labs are building all three. OpenAI has evals, Anthropic and OpenAI have some form of memory. A context layer is the natural next step. Specification stays a human job only until the models get good enough to infer real intent from sloppy prompts, which they can learn by correlating prompts, work done, and work accepted. Taken to its logical conclusion, the labs learn how each organisation and industry works well enough to bypass even them, and start selling outcomes directly to the customers the organisation used to serve.

I am not an AI doomer; I love the technology and believe we can do and build great things with it. But the concentration of this much capability inside a handful of companies is worth worrying about, and worth building against while the knowledge of how to get real work done still belongs to the rest of us.

We have seen this play out in adtech before. Google built an end-to-end system everyone was more than happy to adopt at first. Later we realised this much monopolistic advantage in a market is not good for competition and transparency. So let's not make the same mistake again.