MediumUnderstanding the Cost of Coding Agents | by Alexandra Rusina | Jul, 2026 | Towards AISitemapOpen in appSign up<br>Sign in

Medium Logo

Get app<br>Write

Search

Sign up<br>Sign in

Towards AI

We build Enterprise AI. We teach what we learn. Join 100K+ AI practitioners on Towards AI Academy. Free: 6-day Agentic AI Engineering Email Guide: https://email-course.towardsai.net/

Understanding the Cost of Coding Agents

Alexandra Rusina

8 min read·<br>2 hours ago

Listen

Share

From individual developers running out of tokens on a daily basis to Uber famously burning through the yearly budget in just four months, AI cost in both tokens and dollars is at the center of attention. Even for those who are happy with their AI subscriptions, the API-equivalent cost matters. Once you max out subscription tokens, you can add credits and effectively move into metered usage. Also, Copilot’s shift to metered AI billing raises the possibility that other providers will make the same switch.<br>This article is for those who want to understand how AI billing works in general and how it applies to coding agents. If you are a leader in charge of an AI budget, a TPM figuring out project constraints, or a founder thinking through cost models, this should be a helpful guide. It won’t explain the basic concepts like a “token” or “coding agent”, but it will discuss how tokens flow through the system and show up in your telemetry and billing, as well as how performance and engineering issues can affect your budget.<br>Coding agents and tokens<br>One of the challenges with understanding AI billing is that “token” is a very generic term. You don’t just spend “tokens.” You spend input, cached input, output, reasoning, and sometimes cache-write tokens. These are not interchangeable, because they have different prices and tell you different things about agent and model behavior.<br>Press enter or click to view image in full size

Tokens in the agentic loopThe diagram above shows the overall agentic loop and what you pay for during its execution. Let’s look at each part.<br>Agent/Orchestrator<br>The agent collects client-side data — from the user prompt to file contents and tool call results — and sends it to the LLM via the HTTP request. This step is usually free of charge.<br>Whatever the agent sends to the LLM is called “context” and when you hear discussions about “context window” — this is what it refers to. Context window size is more of an engineering metric than a business metric. It defines how much data the agent can send, but there is no direct billing associated with it. If the provider increases the context window, it won’t necessarily affect your bill.<br>Input/Context Processing and Prompt Cache<br>The AI provider processes the input, and this is where another common term appears: “prompt cache.” The name is a bit misleading, because it’s more of a general context cache. This is not technically part of the LLM, but most AI providers use it these days.<br>The implementation of caching and its billing can differ, but overall it works like this. The data sent from the client always contains the full content, but it still needs to be converted into tokens, or “tokenized”.<br>Because of the agentic loop, the system normally sends the same data over and over again and appends new data at the end. To avoid reprocessing the data multiple times, the providers cache it.<br>Let’s say your agent read five files in the previous step and now added a sixth one. The agent sends all six files to the provider, but five of them will be read from a cache and only one will be considered “fresh input”.<br>Why is this important? Because cached input is typically 10x cheaper than new input. So you will pay the cache token price for five files and the input token price for just one file. The new file will be added to the cache, and during the next agentic cycle you will pay the cached token price for all six files.<br>Cache optimization is a topic of its own. Some providers, like Anthropic, charge separately for writing to it and let you control the amount of time the data lives in the cache. Others, like OpenAI, have fully automated the process and bundled the cost of cache writes into the cost of fresh input tokens.<br>There are two engineering terms you might have heard of: “cache hit” and “cache invalidation”. “Cache hit” means the system finds the old data in the cache and you pay less. “Cache invalidation” means that the data isn’t found, so you pay the input token price.<br>Here is a classic example of how to ruin your cache — add a timestamp at the beginning of the agentic prompt and update it every cycle. While the data in the prompt mostly remains the same, the cache is constantly invalidated. For you, that means paying the input token price for something that has already been cached.<br>Cached input tokens are the cheapest tokens you spend, but remember that you pay the cached price over and over again during the loop, because the agent keeps sending the full context every time.<br>LLM (Model...