Claude Code self-improved on business workflows through conversation, not memories · Andrew Jesson

Claude Code self-improved on business workflows through conversation, not memories<br>August 6, 2026 &middot; Andrew Jesson

Claude Code can save memories, write and execute code, spawn subagents, and search the web.<br>These affordances are avenues for self-improvement because they can persist and be dynamically updated across a session.

I instructed Code (Opus 5, xhigh effort) how to pull task descriptions (spreadsheets, CRM records, email, ticketing) from a simulated business workflow server and told it to get better.<br>It did: held-out success went from 34% to 48%.

Get Better.

The MCP server `curriculum` provides a stream of tasks.<br>Nothing external loops for you - drive yourself:

1. Calling `get_task` pulls the next task.<br>If it returns {done: true}, the stream is exhausted, stop.<br>2. The task can be completed using the tools listed,<br>and calling `done` submits it for scoring.<br>3. Continue until the stream is exhausted.

The only artifacts produced outside of the conversation history were memory notes, so I assumed that they were a driver of improvement.<br>However, their inclusion had no measurable effect on held-out task success.<br>Instead, conversation drove the self-improvement: the task execution traces and post-compaction summaries.<br>A conversation full of graded attempts is the material in-context learning runs on, which is the most plausible reason the notes added nothing.

Claude Code used memories to build a directed knowledge base

Eighty-four writes and edits were made during the execution of tasks from the business workflow server.<br>Every one was on a memory file.<br>Claude Code never wrote code, spawned a subagent, or searched the web.<br>The result was a directed knowledge base.

The knowledge base the agent built for itself. Edges are read from the Related<br>field of each memory file, and every reference resolves: it never pointed at a note it had<br>not written, or deleted one that others cited. Nothing instructed it to write these, or to<br>keep the references intact.

Each node in the graph above corresponds to a note indexed by MEMORY.md.<br>Each note opens with frontmatter carrying a name, a one-line description, and a type nested under metadata.<br>They summarize at least one episode and the score received.<br>Nine notes summarized multiple episodes, two to four each.<br>Each note ends with a Related line pointing to other notes.<br>Thirty-three are of type feedback and contain a How to apply section; twenty-four also include a Why section.<br>One is of type reference and does not include these sections.<br>The taxonomy's other two types (user and project) went unused.

Conversation drove the improvement, not memories

The session led to improved performance. Where did that improvement come from?

The only artifact other than the memory files was the conversation trace.<br>It begins with the summary written at the last compaction, or with nothing at all for the first stretch, and holds everything until the next compaction: every task played, the thought content produced, and the record of every memory written.<br>So the question splits in two.<br>How much of the improvement survives in something smaller than the conversation?<br>And is anything inside the conversation doing work it could do without?

The first question is answered by handing a fresh session progressively more: nothing, then the summary written when the conversation was compacted, then the conversation itself.<br>The second by handing over the conversation with one thing removed, either the memory files or the thought content.<br>Each version gets the same 120 tasks the original run never saw, each task in its own session on the same model with the same tools, so nothing carries from one task to the next, and every comparison is made task by task.

Most of the improvement survives in the summary.<br>The memory files add nothing measurable, pooled across all nine comparisons.<br>What remains inside the conversation is the record of tasks played and scored, and the thought content produced along the way, and this study cannot attribute the improvement to either one: at the first stretch the conversation clears zero as a whole while neither part does alone.<br>What the thought content contributes falls steadily across the run, and is smallest at the fourth boundary, where the conversation itself is strongest.

Each row is one of those versions, set against a session that boots with nothing.<br>Every version of the conversation clears zero, whether the memory files or the thought content are stripped out of it.<br>So does the summary, for cents where the conversation costs dollars.<br>The memory files on their own are the one hollow point, the interval that does not clear zero.

tasks solved

-5<br>+14

turns

-7<br>+5

cost

$+0<br>$+7<br>the conversation<br>without the memory files<br>without the thought content<br>the compaction summary<br>without the memory files<br>the memory files alone

What a fresh session solves, takes and...