How Compaction Works in Pi | EARENDIL
How Compaction Works in Pi
Date:Thu, 13 Aug 2026
From:Earendil Engineering rfc@earendil.com>
To:You
Subject:How Compaction Works in Pi
If you've ever had a long coding session in a coding agent like Pi, Claude Code, or Codex, you will have triggered a compaction.<br>That is because large language models (LLMs) have limited context windows.
The context window is what the model can "see" while producing a response.<br>Transformer architecture limits how much input an LLM can process.<br>The input for a coding agent session includes all the previous messages and tool calls.<br>Hence LLMs reject requests that exceed the context window.
In this post, we will discuss when compaction is needed, and how it works in Pi.<br>Compaction is also a useful tool for managing the size of the context window size, which both help reduce the cost of LLM requests and reduce context rot.
An LLM conversation
When working interactively with a coding agent like Pi, the agents and LLM exchange messages.<br>Each request to an LLM contains initial context including a system prompt, as well as some additional input.<br>This is typically files loaded into the context such as AGENTS.md, and tool definitions.
A coding agent's first LLM request contains this initial context, along with a first user message.
request 1:<br>[system][tools][user]
This starts a turn.<br>The LLM may first return an assistant message containing tool calls.<br>The agent program executes them and sends their results back to the LLM, which can then return another assistant message.<br>The turn is finished when the assistant has completed generating output.
after request 1:<br>[system][tools][user][assistant: tool call][tool result][assistant]<br>returned by LLM | returned by LLM<br>produced by the agent
We continue working, and send another message.
request 2:<br>[system][tools][user][assistant: tool call][tool result][assistant][user]<br>new user message
Each turn expands the conversation.<br>Eventually, the history exceeds the context limit.<br>The next request then returns an error such as Request exceeds the maximum size.
[system][tools][user][assistant][....][tool result][user]<br>exceeds context window
When we cannot continue with the existing conversation as-is, we have two choices.
Handling context overflow
We can start a new, empty conversation without the accumulated context.<br>If we know what we were going to do next, this may often work, but will inevitably lose the conversation history.<br>It might still be a good idea to do, because it is observable that the performance of LLM outputs decrease as the context size grows.
We can create a smaller representation of the conversation context, since we want to keep this conversation going.<br>That is what compaction does.
Compaction
In theory, there are many ways to implement compaction.<br>For example, we can write a deterministic function which keeps some of what is in the conversation and discards the rest.<br>In practice, though, implementations of compaction use an LLM request to summarize the conversation history.
Regardless of the method, after compaction, the context should have been compressed such that we have room for many new messages and tool calls.
[system][tools][compaction result][user]<br>new message
Pi's implementation
Let's look more closely at how Pi specifically implements compaction.
When conversations grow too long, Pi uses compaction to summarize older content while preserving recent work.<br>Compaction is triggered when the context limit is nearing the total size of the context window.<br>It can also be manually triggered using the /compact command.
Pi checks for auto-compaction after a turn ends.<br>Until then, each request extends the existing prompt and can reuse its cached prefix.<br>Pi may also compact mid-turn, if it encounters a context overflow error.
When compacting, Pi retains some number of recent messages unchanged.
before compaction:<br>[system + tools][older turns][recent retained messages]
How many messages are retained vary by session, but it's determined by a configurable number of tokens.<br>Pi's current default of 20 thousand tokens comes out to roughly 5-20 turns.
All the messages before this cut point are extracted and serialized, and will be summarixed.<br>To keep the compaction request within the context limit, Pi truncates tool call results in the history to 2,000 characters.<br>If we didn't somehow reduce some of the history, we would already be above the context limit.<br>Tool outputs are a reasonable place to cut because they have a more intermediate nature.
The compaction request that Pi sends differs from regular conversational requests.
The system prompt used in the standalone compaction request is different.<br>Instead of telling the LLM "you are an expert coding assistant", we tell the LLM "you are a context summarization assistant".
The user message in the compaction request is also different.<br>It requests "a structured summary of this conversation branch for context when...