Measuring token compression for coding agents on SWE-bench Lite - Edgee Blog<br>Login

Measuring token compression for coding agents on SWE-bench Lite<br>Khaled Maâmra<br>Research Engineer

July 2, 2026Engineering

Why compression matters ?

Coding agents have become the substrate for building products. They are long-running and context-heavy. For example, a single SWE-bench coding task can take 30–100 API turns, where 1–10 million tokens per task is common. These numbers, multiplied across a fleet of agents, a developer team, or a SaaS product make the bill escalates quickly in the blink of an eye.

The economic pressure is one motivation for compression, but not the only one. Other elements point in the same direction:

Bottleneck<br>What it limits<br>Why compression helps

Dollar cost<br>Per-task spend; sustainable unit economics for AI products<br>Fewer tokens (of the right kind) → smaller bill

Latency<br>Time-to-first-token scales with prefix size on most providers<br>Smaller prefix → faster perceived response

Context window<br>200k or 1M token windows still fill up on long agent sessions<br>Compressing history extends the effective horizon

Throughput<br>Server-side batch sizes are inversely related to per-request token count<br>Smaller requests → more parallelism for the provider; lower queueing delays for the user

Hence, a robust compression layer addresses all four and is therefore a critical part of an agentic workflow performance. Compressor V2 is such a layer and is a component of the Edgee AI gateway that was built with these requirements in mind.

From V1 to V2

Edgee's V1 compressor, shipped earlier this year, used a single strategy we called tool result trimming. This strategy is inspired by the famous RTK project and consists in cleaning up the verbose tool outputs that agents collect over a long session. It delivered roughly 10% cost savings on real coding workloads — safe, simple, but limited in scope.

V2 takes a different approach: three orthogonal strategies layered together , each one targeting a different layer of context and configurable independently per API key. Since the three strategies don't overlap they attack different sources of token bloat:

Brevity attacks the most expensive token class, output tokens.

Tool surface reduction (TSR) attacks the most repetitive part of the prefix (MCP tool catalogs).

Tool result trimming attacks the long tail of verbose tool outputs that accumulate over the conversation.

Through the Gateway, customers compose the combination that matches their workload and reduce their token consumption. In this post, we address the empirical results that quantify exactly these gains.

To start, let us recall some important notions.

Prefix caching

Anthropic's prefix cache is content-keyed, any byte that differs between two requests invalidates the cache from that point on. A long session that reuses the same system prompt and tool catalog pays the cache_create fee once and reads back the prefix at 1/10th the input price on every subsequent turn.

This is also why strategies like brevity or tool result trimming are so well-suited for coding agents. They touch only the output, never the prefix. Therefore, cache amortization across long agent sessions stays intact, and the savings are concentrated in the most expensive class.

Statistical tools

For every metric we report, three things are computed:

Paired sign test. For each task, compare per-task means: did edgee use less? Count the wins. The win count follows a binomial(n, 0.5) under the null and p Bootstrap 95% Confidence Interval. Resample task pairs 10,000 times, compute the statistic on each resample, take the 2.5th and 97.5th percentiles.

Within-task coefficient of variation. Average across (task, backend) cells. Above 20% means too much per-replicate noise; below 20% means the result is trustworthy.

Statistical methodology in depth

This section walks through the tests we apply, why we chose them, and what they tell us.

The structure of the data

For every (task, backend) combination, we collect N replicate measurements and average across replicates to get a single per-task mean for each backend. The result is a paired dataset: n tasks, each with one vanilla mean and one edgee mean. Pairing matters — task difficulty varies by orders of magnitude (some SWE-bench tasks are 100k-token toy fixes, others are 12M-token deep refactors), so unpaired comparisons would be dominated by between-task variance.

The paired sign test

Our primary tool for direction-of-effect claims.

Setup. For each of n tasks, ask: did edgee use less cost than vanilla? Count the yeses. Under the null hypothesis ("no real difference between backends"), the count of edgee wins follows a binomial distribution with parameters n and probability 0.5.

Two-sided p-value. Given k wins out of n:

Pr<br>max(<br>Binomial(<br>0.5)

For n = 6 tasks and k = 6 wins: p = 2 · (1/2)⁶ = 0.031. The brevity result for example lands in this area.

Why this test, specifically. We chose the...