How Prompt Tuning Improved GPT-5.5 in VS Code
📼 Rewatch VS Code Live at MS Build 2026
Dismiss this update
How Prompt Tuning Improved GPT-5.5 in VS Code
July 6, 2026 by VS Code Team, @code
In our previous post, we introduced the VS Code coding harness, the layer that connects the model to tools, context, instructions, and the agent loop, giving the model the ability to perform coding tasks.
Each model responds to tool calls and instructions differently, and the harness can adapt to improve results. This post walks through a two-week experiment we ran in partnership with OpenAI to tune the GPT-5.5 system prompt in VS Code. The question was simple: if we nudge the agent to explore less and validate sooner, can it get faster and cheaper without getting worse? With OpenAI's model expertise and our harness data, we tested two small prompt changes, measured them against a control on live traffic, and shipped the winner.
This matters more with usage-based billing in place. Token efficiency isn't only an infrastructure metric: every token the agent spends wandering is a token you pay for and wait on. An agent that reaches a grounded edit sooner is both a better experience and a smaller bill.
The hypothesis: explore less, validate sooner
Following the launch of GPT-5.5, we looked at how the model spent tokens inside the VS Code agent harness, as part of the work described in Improving token efficiency in GitHub Copilot. Two patterns stood out: where the model spent tokens, and where it over-explored before acting. Agents can spend a lot of effort searching, rereading, and comparing nearby paths before making a useful edit.
That pointed to a single, testable idea: the agent should spend less effort wandering and more effort moving through a deliberate loop of evidence, action, and validation.
After testing different hypotheses and running offline evaluations, we turned that idea into two variants of the GPT-5.5 system prompt, both were promising in offline evals, and we tested them against the current default on live traffic.
Inside the experiment
We ran the experiment in VS Code over a two-week window, splitting GPT-5.5 agent traffic across two treatment groups and one control group with a 25/25/25 split. Both treatments test the same hypothesis but differ in how much structure they add to the prompt.
Group<br>Variant name<br>Description<br>Traffic allocation
Control<br>PRPT_CTRL<br>Current default prompt<br>25%
Treatment A<br>PRPT_SRCH<br>Economical search and edit: single, compact reminder to limit exploration before acting<br>25%
Treatment B<br>PRPT_LRG<br>Large prompt sections: broader restructure covering the full edit-and-validate loop<br>25%
Note: The allocations add up to 75% because the experiment scorecard compares evenly sized groups. The remaining GPT-5.5 traffic continued to use the default prompt outside this scorecard slice, so we could compare the treatments and control across the same kind of user traffic.
Treatment A: economical search and edit
Treatment A makes a small, focused change: a single, compact reminder that nudges the model to reduce unnecessary exploration.
The section in the prompt instructs the agent to start from a concrete anchor, gather only enough local context, avoid broad exploration, act once there is a cheap discriminating check, and avoid rereading unchanged context.
You can find the complete implementation details in gpt55BasePrompt.tsx:
{economicalSearchAndEditEnabled && Tag name='economical_search_and_edit'><br>- Start from the most concrete available anchor: a file, symbol, failing behavior, failing command, or nearby implementation surface.br /><br>- Gather only enough nearby context to choose one plausible local hypothesis and one cheap check that could disconfirm it.br /><br>- Prefer one targeted search or nearby read over broad repo exploration.br /><br>- Once the cheapest discriminating check is known, act.br /><br>- Do not re-read unchanged context unless a new result makes it relevant.br /><br>Tag>}
Treatment B: large prompt sections
Treatment B tested a broader version of the same idea of limiting exploration. Instead of adding a single, compact reminder about economical search, it reorganizes the agent workflow into explicit and sections. Unlike Treatment A, these additions make the system prompt itself larger, so a key question was whether the added structure would still improve efficiency, not just agent behavior.
The goal was to solve the full loop and not only the search step: form a local hypothesis before editing , avoid broad exploration , make a grounded first edit , and validate immediately after the first substantive edit .
You can find the complete implementation details in gpt55BasePrompt.tsx:
{largePromptSectionsEnabled && <><br>Tag name='Before_the_first_edit'><br>- Start from the most concrete anchor available: a file, symbol, failing behavior, failing command, test, or nearby implementation surface. If the request does not name one explicitly, use the first targeted search or...