How Compaction Works in Pi | EARENDIL
How Compaction Works in Pi
Date:Thu, 13 Aug 2026
From:Earendil Engineering rfc@earendil.com>
To:You
Subject:How Compaction Works in Pi
If you have ever had a long coding session in a coding agent like Pi, Claude Code, or Codex, you will have triggered a compaction.<br>In this post we explain how compaction works and when Pi needs to compact.
An LLM conversation
Large language models (LLMs) have limited context windows.<br>The context window is what the model can "see" while producing a response.<br>The transformer architecture used by LLMs limits how much input they can process.<br>The input for a coding agent session includes all the previous messages and tool calls, and this keeps growing as you work.<br>Once it exceeds the context window, the LLM rejects the request.
When working interactively with a coding agent like Pi, the agent sends requests to an LLM and receives responses.<br>Each request includes a system prompt, loaded files such as AGENTS.md, tool definitions, and the conversation history.
A coding agent's first LLM request contains this initial context, along with a first user message.
request 1:<br>[system][tools][user]
This starts a turn.<br>The LLM may first return an assistant message containing tool calls.<br>The agent program executes them and sends a new request to the LLM containing the complete conversation, now including the tool results.<br>We get back another assistant message.<br>The turn is finished when the assistant has completed generating output.
after request 1:<br>[system][tools][user][assistant: tool call][tool result][assistant]<br>returned by LLM | returned by LLM<br>produced by the agent
We continue working, and send another message.
request 2:<br>[system][tools][user][assistant: tool call][tool result][assistant][user]<br>new user message
Each turn expands the conversation.<br>Eventually, the history exceeds the context limit.<br>The next request then returns an error such as Request exceeds the maximum size.
[system][tools][user][assistant][....][tool result][user]<br>exceeds context window
Handling context overflow
When we cannot continue with the existing conversation as-is, we have two choices.
We can start a new, empty conversation without the accumulated context.<br>This discards the history, including prior decisions and unresolved work.<br>It might still be a good idea to do, because the performance of LLM outputs decrease as the context size grows.
We can create a smaller representation of the conversation context, since we want to keep this conversation going.<br>That is what compaction does.
Compaction
In theory, there are many ways to implement compaction.<br>For example, we can write a deterministic function which keeps some of what is in the conversation and discards the rest.<br>In practice, though, implementations of compaction use an LLM request to summarize the conversation history.
Compaction replaces part of the history with a compressed representation, leaving room for additional messages and tool calls.
[system][tools][compaction result][user]<br>new message
Pi's implementation
Let's look more closely at how Pi specifically implements compaction.
When conversations grow too long, Pi uses compaction to summarize older content while preserving recent work.<br>Compaction is triggered when the context limit is nearing the total size of the context window.<br>It can also be manually triggered using the /compact command.
Pi checks for auto-compaction after a turn ends.<br>Until then, each request extends the existing prompt and can reuse its cached prefix.<br>Pi may also compact mid-turn, if it encounters a context overflow error.
When compacting, Pi retains some number of recent messages unchanged.
before compaction:<br>[system + tools][older turns][recent retained messages]
The number of retained messages varies because Pi uses a configurable token budget.<br>Pi's current default of 20 thousand tokens comes out to roughly 5 to 20 turns.<br>All the messages before this cut point are extracted and serialized, and will be summarized.
Pi's compaction prompt
The ideal outcome of a good summarization for a coding agent is like a handoff briefing from one shift to the next.<br>Pi's compaction prompt focuses on the fact that there is a lot in the existing context that is no longer relevant.<br>We should only keep around what is still important context for the next LLM request.
Pi therefore sends a different request for compaction than for regular conversation.
The system prompt used in the standalone compaction request is different.<br>Instead of telling the LLM "you are an expert coding assistant", we tell the LLM "you are a context summarization assistant."
The user message in the compaction request is also different.<br>It requests "a structured summary of this conversation branch for context when returning later."<br>The prompt specifies sections for goal, progress and key decisions.
It's a standalone request that doesn't use any of the existing conversation history, which means it can...