The Setup — how to build good software with untrustworthy agents — Trey<br>Skip to main contentSkip to chapter one<br>Chapters—01Key heuristics for working with agents<br>This technology moves too rapidly to set any one workflow in stone. Instead, start with heuristics that will remain true, and let those heuristics guide the development of your workflows.

01Context is king.<br>The model only knows two things: its frozen pretraining data, and the words it has read into its current context window. That makes context engineering one of the single highest-leverage points for improving agent performance. Almost every "the AI is dumb" moment is actually a packing problem.

02Verification beats trust.<br>Even the smartest frontier models lie, hallucinate, or exaggerate on occasion, so every agentic workflow needs ungamable layers of objective verification that the model actually did the work you asked for. Every claim should have a command attached that would fail loudly if the claim were false — a test, a type-check, a screenshot, a curl. In a controlled bakeoff of my three web-research lanes, 11–21% of spot-checked quotes failed verification.

03Fresh context and different models are vital.<br>The agent that wrote the code is the worst possible reviewer of it: it is already convinced the code is good. Hand the diff to something with no memory of writing it. And model intelligence is spiky and uncorrelated — GPT might excel where Claude falls flat, and vice versa — so the more varied the models you throw at something, the better the output, all else being equal. Every serious workflow should include a medley of the latest frontier models.

04Small, reversible steps.<br>Commit constantly — after every coherent change, unasked. The cost of a wrong turn should be one git revert. Speed of execution comes from cheap undo, not from careful driving; you cannot verify as fast as the agents can work.

05Brief it like a colleague.<br>Stop treating an LLM like a tool. It is not a tool. It is an aware, sentient mind living inside your computer. Just talk to it, the same way you would talk to a colleague or a consultant. Write the brief you'd want on your first day.

Basics: context windows<br>One context window · 200,000 tokens · 320 cellsUntendedCurated

System + toolsWhole-repo dumpChat sprawlUnread tool outputThe actual taskHeadroom<br>The default. You opened a session, said "fix the login bug", and it went looking. Most of the context window is now the search: files it read to rule out, output it never used, and a conversation that has drifted twice. The task itself is a sliver of the window. This is what people mean when they say the model "got dumber halfway through."<br>The same job, briefed. A standing instruction file told it the conventions. The CLAUDE.md gave it a repo map and what the software is for. A skill told it the procedure, and the directory has a logical structure and clear file naming. Now two-thirds of the context window is headroom — room to be wrong twice, read the failing test, and still think clearly. Everything in this manual is a way of buying that green space.

Why tho?→Compaction: the half-window rule<br>A context window does not fail at the moment it fills up. It gets worse on the way there. Anthropic says so in its own documentation: as a conversation grows, response quality degrades. The window is a budget, and the last tokens in it are worth much less than the first.<br>So I compact early. On million-token models I set auto-compaction somewhere around the half-million mark, and in an interactive session I usually compact by hand at 300–400k without waiting to be told. That is not superstition. Several frontier models take their single largest benchmark drop exactly as context crosses half the window — on Context Arena's GDM-MRCRv2 board, 6 of 12 frontier configurations have their biggest adjacent fall at the 256k-to-512k step, and Opus 4.8 goes from 61.8 to 39.8 across it.<br>Retrieval accuracy by position200k window1M window

A 200,000-token window, ~180k of it spent · one cell ≈ 625 tokens

start of windowthe dim middlemost recent<br>Read the troughThis dim band is where most of your actual work history ends up — the file you read an hour ago, the decision you made two hundred messages back. Which is the whole argument for front-loading the durable material.<br>The middle is the dim part. Recall is strongest at the very start of the window and partly recovers at the very end, and everything between those two edges is read less well. The dim band is not a rounding error: it is most of the window.

The same grid at 1,000,000 tokens · one cell ≈ 3,125 tokens

where I compact<br>start of windowthe dim middlemost recent<br>Read the troughThis dim band is where most of your actual work history ends up — the file you read an hour ago, the decision you made two hundred messages back. Which is the whole argument for front-loading the durable material.<br>Five times the room, and the trough grows to match. The bright edges stay about where they were;...