Drop one tool from your request: one GPT-5 version keeps 76% of it cached, another keeps nothing | Srce Cde Skip to content<br>Drop one tool from your request: one GPT-5 version keeps 76% of it cached, another keeps nothing<br>Jul 22, 2026 · 21 min read · By Chirag (Srce Cde)

Prompt caching is the cheapest performance win within an LLM based application. An application sends a long system prompt with each request and the model provider checks recognizes the part it had seen before (from the beginning) and it charges you a fraction of actual token cost instead of reprocessing them. An agent that uses tools where the system prompt and tool definitions can be upto several thousand tokens and barely change between turns, the discount is enormous.

The advice to benefit from the caching is simple - keep the front of your prompt stable. Have the static content first, variable content last, and freeze your tool definitions so the cached prefix stays intact. The advice is correct but it is also version-dependent in a way nobody mentions , and the dependence is severe.

I ran one experiment by dropping a single tool from the request and changing nothing else across four versions of the same model family for fifteen times per version, in three batches across two days. On gpt-5.2 the change cost me a quarter of my cache. On gpt-5.5, two versions later, the identical change threw away the entire cache, system prompt included. And on gpt-5.1, the same test wouldn’t settle on one answer, which turned out to be a finding of its own and its the reason that this post reports replicated runs instead of single ones.

This post is the experiment, its results and what they mean for anything that changes its tool list at runtime.

Here is the short version, before the method. The same operation on every row - remove one tool, percent of the request still served from cache:

modeldrop the last tooldrop the first toolgpt-5.276%46%gpt-5.4~80%0% gpt-5.50% 0%gpt-5.1wouldn’t settle on one answer so it has its own section below<br>The rest of the post is how these numbers were measured, why they can be trusted and what to do about them. The whole experiment runs on a small synthetic rig which you can copy.

You’ll get the most out of this if you’re already comfortable with:

how a chat completion request is structured - a system message, tools definitions and a user message

the idea that the model sees your request as one flat sequence of tokens, not as separate fields

roughly what prompt caching does (charges less for a repeated prefix). If that’s new to you then the OpenAI and Azure caching docs are a fine ten-minute primer first

Caching is a prefix match

Prompt caching is not semantic matching as the providers do not lookup by context instead it walks your token sequence from the first token, left to right and then finds the longest run that is byte-for-byte identical to something already in its cache. That longest byte-for-byte identical sequence is the prefix and everything after the identical sequence mis-match/break is processed as fresh, at a full price.

Position is everything - A change near the begining of your prompt invalidates the front and everything after it because the match stops at the first difference. A change near the back invalidates only the back.

Identical-but-unreachable content is worthless - If two requests differ at token 50, then the 3,000 identical tokens sitting at position 51 onward cache nothing as they are past the break.

Size floor - Providers only cache a prefix once it clears a minimum length of 1,024 tokens on the OpenAI/Azure models. Anything below that, you get no caching, no error, just a bill.

Synthetic agent with nothing proprietary in it

To measure this cleanly I needed an agent shaped request that I could publish. So the domain is a fictional public library catalogue. Structurally it mirrors a real tool using agent, and three properties that are baked in:

Long static system prompt (~1.8k tokens) - the analyst instructions, workflow rules, formatting rules.

Small varying block near the top - a few lines describing which collection we’re working over. It carries counts, not lists, so it is nearly constant width and differs between collections only in its digits. This is an analogue of a per-user or per-dataset block.

8 tool definitions - passed in as tools= which are deliberately verbose so the block clears the floor of 1,024 tokens on its own (~1.5k tokens).

One measurement that matters comes from the API itself:

def cached(resp) -> int:<br># prompt_tokens_details can arrive as a plain dict on older SDKs,<br># so read it dict-safely rather than via attribute access.<br>d = resp.usage.model_dump().get("prompt_tokens_details") or {}<br>return d.get("cached_tokens", 0) or 0

cached_tokens is how many tokens of this request were served from cache. That single number is the whole experiment.

Note

On older SDK versions, prompt_tokens_details is returned as a raw dict instead of a typed object, so...