Don't Put an LLM Behind an MCP Server

Don't Put an LLM Behind an MCP Server | Ryan Spletzer

Giovanni Domenico Tiepolo, Public domain, via Wikimedia Commons.

Hi-diddly-ho, neighborino! Greetings from the AI ice cream shop, where I’m sad to say they’re all out of the tokenmaxxing flavor, and the new flavor of the day is tokenomics.1

But tokenomics is the subject of another blog post.

Today’s post is about some nerdy technical stuff, so if that’s not your jam,2 feel free to bop around to another part of the internet, but I promise you there is a gem here worth considering, as well as several solid pop culture references, as usual.

With that, let’s dig in…

Suffice it to say, I’ve had enough of some of the shenanigans people have gotten up to with MCP servers lately.

I recently tried a CLI tool from a vendor (who shall not be named, but you have definitely heard of them), which essentially wraps an MCP server (so you can either use this tool via CLI or via MCP directly). I called the CLI from Claude Code, which is a recommended way to use this tool. I asked a simple question about some simple data that lives in the vendor’s system behind it all, the kind of lookup that any reasonable API answers in a few hundred milliseconds (and the vendor’s own native APIs are exceedingly good at answering quickly). Instead, the CLI’s request to its backing MCP server waited on a hidden thinking loop, and I sat there watching my local agent wait for another hidden remote “agent” to think, for double-digit seconds, long enough to ponder the entirety of the meaning of life.

The frustrating part is that I saw this coming. Many months ago I was writing an engineering strategy, and in it I warned our teams that putting an LLM behind an MCP server was the wrong impulse, because it would be hella slow.3 At the time I could only predict the issue on paper. This vendor’s CLI confirmed my fears in practice, and now I point to it as a live example of what not to do.

Putting an LLM behind a Model Context Protocol (MCP) server, then calling it from an MCP client that is itself an agent with its own LLM, breaks the social contract of what MCP was designed to be for users of these tools: a way for agent clients to query data and provide context to the model. It was never meant to facilitate communication with yet another model or agent. Protocols for exactly that have since emerged—A2A and ACP among them—but nobody driving a Claude Code session that calls out to a CLI or an MCP server is expecting an LLM on the other end of that call.

Somewhere along the way, some of us in this industry lost our minds. I sure hope we find them.

Don’t do this. Don’t put an LLM behind an MCP server.

The anti-est of patterns

The anti-pattern looks like this: an MCP server exposes a tool, but the tool’s implementation is feeding a prompt to another model. Your agent calls something like get_insights(question), and behind the curtain a second model—one you didn’t choose, running a system prompt you can’t read, over context you can’t see—generates prose and returns it as the tool result. Sometimes it goes further, and the “tool” is a whole agent with its own loop, tools, and retries.

It’s matryoshka nesting-doll architecture (and not in a good way): you open the tool and there’s another model inside.

It doesn’t help that people confuse MCP servers with agents. AI hype swirls, and in a weird way it works itself into real architecture: a leader erroneously asks for an “MCP agent,” a development team living under a rock vibe codes the misinterpretation, and now they have a Frankenagent on their hands.

On a slide, this looks and sounds sophisticated. Why settle for a dumb pipe when the server itself can think? In practice it breaks the contract that makes MCP useful in the first place. Ian Malcolm had the diagnosis back in 1993: your engineers were so preoccupied with whether or not they could, they didn’t stop to think if they should.

MCP delivers data, not decisions

The whole point of MCP is to deliver context from external systems and let the client agent drive: the agent decides what it needs, expresses that intent through tool parameters, gets results back, and does the reasoning itself. Tools work best when they behave like queries and commands—parameterized, predictable, and inspectable. I’ve written before about applying CQRS’s ask-versus-act split to agent tools, and the same discipline applies here: a tool should either fetch something or do something, and in both cases the agent should be able to trust what came back.

An LLM behind the tool inverts that relationship. The server now decides what your agent gets to know. A database row is checkable. A search result carries its source with it.4 A hidden model’s summary is an opinion wearing data’s clothes, and your agent will treat it as ground truth, because tool results land in the context window with the authority of being a “fact.”5

Two models, compounding failure

Egon Spengler’s rule applies here: don’t...

Don't Put an LLM Behind an MCP Server

Related Articles

(no title)

Scientists reverse brain aging, with a nasal spray

AI has torched the market for junior programmers

Al Vigier: Canada's AI strategy shouldn't include secret Palantir bills

The Anatomy of an AI-Native Org