Ace Sidecar – Efficiency Optimization for Local AI Coding

flyingfishisme1 pts1 comments

Announcing the ACE sidecar: observability for coding-agent sessions — ACE Blog← /blogAnnouncing the ACE sidecar

Coding agents are the least observable expensive thing most engineering orgs now run.

You know your monthly API bill. You do not know which sessions produced it, how much of it was<br>context re-read for the twentieth time, how long your agents spent parked waiting on an approval<br>nobody was there to give, or whether any of the optimisations on your roadmap would be worth a<br>dollar on your traffic.

We spent five weeks measuring one developer's Claude Code sessions request by request, and<br>published what we found: 4.2 billion prompt tokens,<br>$3,035 at list price, 90.5% of it input that had already been sent , and — more usefully — a<br>string of levers that looked large in theory and collapsed under measurement.

The instrument we built to do that is now a thing you can run.

The ACE sidecar is a local reverse proxy that sits between your coding agent and the model<br>provider. It relays your traffic unchanged, prices every request, reads your existing transcripts<br>off disk, and renders the whole picture on a dashboard at 127.0.0.1. No account. No upload. No<br>change to how you work beyond one environment variable.

// sidecar_beta<br>Get beta access<br>The sidecar runs entirely on your machine and uploads nothing. Leave an address and we'll send you the build and the quickstart when the beta opens — or just to stay in the loop.<br>Email addressGet beta accessWe use it to send you the beta and nothing else. Unsubscribe in one click.

What it shows you

Everything below is a real dashboard, rendered from real session data. Paths, project names,<br>prompts and session identifiers have been replaced with synthetic equivalents — the aggregate<br>numbers are untouched.

The eleven numbers that describe your fleet

Tokens in and out, requests, handbacks to the user, cost per turn, cost per session, commits per<br>session, tokens per commit. The distribution table underneath is the one that usually lands: the<br>p25 request already carries 100k+ tokens of context . There is no cheap quartile — which is why<br>per-request payload tuning loses to context-size work, every time.

Commit attribution is time-window based and says so on the page. A commit that landed inside a<br>session's window was not necessarily produced by that session, and the dashboard will not pretend<br>otherwise.

The shape of the work

Daily tokens, daily commits, idle days. On this corpus the day-over-day median change is −12.2%<br>with an interquartile range of −48% to +76% — volume routinely halves or doubles between<br>consecutive days.

That single chart kills a category of work. Any alert or budget built on a rate of change will<br>fire constantly and mean nothing. Only a cumulative cap against a period budget survives this<br>shape, and it is much easier to believe that when you are looking at your own week.

Where the money goes, with the rate card attached

Cost, cache savings, cache hit ratio, peak context — then the rate card and the actual formula ,<br>because a page that tells you what something cost owes you the arithmetic. Every rate is read from<br>a versioned catalog and links to the vendor's pricing page with the date it was checked.

Two things people usually notice here. Cache savings are larger than the bill — 6.1× on this<br>corpus — and that saving is the provider's, already banked before any tool of ours arrives; the<br>dashboard labels it that way rather than claiming it. And cache reads are ~98% of prompt volume ,<br>which is the fact the rest of the page is about.

What optimisation would actually be worth, on your traffic

This is the section we most wanted to exist. Two scorecards, because they are two different<br>problems :

Enterprise — minimise dollars. Accounting levers count, because they change nothing the model<br>sees.

User — maximise headroom under a token cap. Accounting levers are excluded, because they<br>convert price without removing a single token.

The per-lever table carries a risk column , and the ordering it produces is uncomfortable: the<br>largest lever on this corpus (ttl_keepalive, 6.4% of cost) removes zero tokens , while the<br>levers that actually remove tokens are small. Everything simulated is badged SIMULATED. We would<br>rather ship a scorecard that says "this is a model with stated assumptions" than one that quietly<br>implies measurement.

Your sessions, on disk, read and never written

The transcripts the numbers came from — main sessions and the subagent runs they spawned, with<br>turn counts, sizes and the first prompt of each. Working directory is read from the transcript<br>itself rather than reconstructed from the directory name, because a project whose name contains a<br>dash cannot be recovered from Claude Code's slug.

The other axis: where the clock went

Cost is one measure of efficiency. Elapsed time is the one developers actually feel, and it<br>decomposes completely differently: 87.7% of wall clock is idle , and 233.8 hours of that — 204<br>occurrences, 69 minutes...

tokens from session cost sidecar sessions

Related Articles