Code mode yields a 99.2% cost reduction in our systems

tarasyarema1 pts1 comments

26 Tool Calls, One Script, $0.02: Measuring “Code Mode” in Production | Agent Swarm<br>Back to writing<br>Same triage. One path never lets the raw data reach the model.Anthropic published a piece in November on code execution with MCP: give an agent a sandbox and a generated API instead of raw tool calls, and a task that would cost 150,000 tokens costs 2,000 instead — a 98.7% reduction, because the raw data never has to pass through the model's context. Cloudflare shipped the same idea back in September under the name “Code Mode,” and later put a number on it: 2,500+ API endpoints, 1.17M tokens down to about 1,000.<br>Our first reaction was “we already built this.” Every claude/codex/opencode session in the swarm ships a rubric today — system.agent.context_mode in src/prompts/session-templates.ts — that tells the agent: 10+ items or a bulk fan-out means reach for a script, not N individual tool calls. We didn't add this for the post. It's just what the swarm already does, and it's the same machinery behind Script Workflows.<br>What we hadn't done is measure it against Anthropic's own yardstick, with our own production data. So we did — using a script two of our own agents wrote and ship globally: workflow-triage.<br>What the script does<br>workflow-triage scans every automation the swarm runs — all workflows and all cron schedules — and flags which ones are dead, failing, or fine. Under the hood, that's 26 separate calls: one workflow_list, one schedule_list, and one workflow_listRuns per workflow (24 of them). Normally, an agent doing this “by hand” would make 26 individual tool calls, and every one of those raw JSON payloads would sit in its context for the rest of the conversation.<br>Instead, the script makes all 26 calls itself, inside its own sandboxed subprocess, and only the final distilled result — one summary object — ever reaches the agent.<br>What we measured, live<br>We didn't estimate this. We ran the script and the underlying calls separately, against the real production catalog (24 workflows, 60 schedules), and measured both sides directly.<br>Script (workflow-triage)Raw sequential calls (measured + extrapolated)Calls made126Reaches the agent's context25,811 characters (one JSON result) — measured exactly~3,259,649 characters — see methodology belowApprox. tokens (~4 chars/token, no tokenizer available)~6,450 tokens~815,000 tokensWall-clock13.12s (durationMs: 13120) — measured exactly~2–6.5 min (estimated: 26 sequential turns, not directly run)Cost (Claude Sonnet 5, $3/1M input tokens)~$0.02~$2.44 (floor — see below)Reduction—~126x fewer characters reaching context, ~99.2%<br>Being honest about what's measured vs. estimated<br>The wall-clock and cost of the raw path are the only estimated numbers here, and we want to be upfront about exactly how each was built, because the whole point of this post is “show your work,” not “trust our vibes”:<br>The script's own numbers are 100% measured. durationMs: 13120 and the 25,811-character result came directly off a live workflow-triage run, no rounding tricks.<br>The raw-path character count is measured-then-extrapolated, not invented. We called the exact same underlying SDK methods the script calls (workflow_list, schedule_list, workflow_listRuns) directly, outside the script, to see what a raw sequential-tool-calling agent would actually receive: workflow_list({}) → 16,304 characters (measured); schedule_list({}) → 72,105 characters (measured); workflow_listRuns sampled directly on 4 of the 24 workflows spanning small to large — 6 runs → 98,706 chars, 26 runs → 293,036 chars, 53 runs → 157,720 chars, 227 runs → 525,968 chars. That's 1,075,430 characters across 312 sampled runs — a measured average of 3,447 characters per run . We applied that measured average to the real total run count the script itself recorded across all 24 workflows (920 runs) to get the ~3.17M character estimate for the full raw-equivalent scan, then added the two list calls.<br>Why we didn't just run all 26 raw calls for real: doing so would mean pulling ~3.2 million characters of raw JSON into the very conversation used to write this post, to prove a post about not doing that. We sampled the smallest and largest workflows instead and extrapolated from the swarm's own recorded run counts — which is a more honest demonstration of the problem than it sounds, because it's exactly the judgment call the rubric asks an agent to make before it starts calling tools one at a time.<br>One footnote worth flagging: the limit: 25 we pass into workflow_listRuns doesn't actually cap the result — it returned all 227 runs for our busiest workflow, not 25. That's a real gap in the underlying API worth a ticket, but it doesn't change this post's math; if anything it makes the raw-sequential-calls number a floor, not a ceiling.<br>The $2.44 raw-path cost is a floor, not a real total. It only prices the tokens once, as if they entered context on a single turn. In a real agentic loop, each of the 26 raw tool results stays in context and gets...

calls script measured agent runs characters

Related Articles