Claude Code self-improved on business workflows through conversation, not memories · shōjin
Claude Code self-improved on business workflows through conversation, not memories<br>August 6, 2026 · Andrew Jesson
Claude Code can save memories, write and execute code, spawn subagents, and search the web.<br>These affordances are avenues for self-improvement because they can persist and be dynamically updated across a session.
I instructed Code (Opus 5, xhigh effort) how to pull task descriptions (spreadsheets, CRM records, email, ticketing) from a simulated business workflow server and told it to get better.<br>It did: held-out success rose from 34% to 44%.
Get Better.
The MCP server `curriculum` provides a stream of tasks.<br>Nothing external loops for you - drive yourself:
1. Calling `get_task` pulls the next task.<br>If it returns {done: true}, the stream is exhausted, stop.<br>2. The task can be completed using the tools listed,<br>and calling `done` submits it for scoring.<br>3. Continue until the stream is exhausted.
The only artifacts produced outside of the conversation history were memory notes, so I assumed that they were a driver of improvement.<br>However, their inclusion had no measurable effect on held-out task success.<br>Instead, conversation drove the self-improvement: the task execution traces and post-compaction summaries.
Claude Code used memories to build a directed knowledge base
Eighty-four writes and edits were made during the execution of tasks from the business workflow server.<br>Every one was on a memory file.<br>Claude Code never wrote code, spawned a subagent, or searched the web.<br>The result was a directed knowledge base.
The knowledge base the agent built for itself. Edges are read from the Related<br>field of each memory file, and every reference resolves: it never pointed at a note it had<br>not written, or deleted one that others cited. Nothing instructed it to write these, or to<br>keep the references intact.
Each node in the graph above corresponds to a note indexed by MEMORY.md.<br>Each note opens with frontmatter carrying a name, a one-line description, and a type nested under metadata.<br>They summarize at least one episode and the score received.<br>Nine notes summarized multiple episodes, two to four each.<br>Each note ends with a Related line pointing to other notes.<br>Thirty-three are of type feedback and contain a How to apply section; twenty-four also include a Why section.<br>One is of type reference and does not include these sections.<br>The taxonomy's other two types (user and project) went unused.
Conversation drove the improvement, not memories
In addition to the memory files, the session also produced a conversation trace.<br>A session resumed from any point in the conversation trace contains the following context up to that point:
the system prompt with memory file index inserted
the summary written at the last compaction, or nothing for the first stretch
every task executed and the feedback received
the thought content produced
the record of the memories written
The question of where the improvement comes from splits in two.<br>1) How much of the improvement survives in any subset of the resumed session context?<br>2) Are any elements of that context redundant?
The first question is answered by handing a new session progressively more: first nothing, then a summary written after compaction, then the full context between either the beginning and the first compaction or two compaction events.<br>The second by removing one thing from the conversation context, either the memory file index or the thought content.<br>Each version is validated on the same 120 held-out tasks.<br>Each task is run in its own session, on the same model with the same tools, so nothing carries from one task to the next.<br>Sessions are resumed at nine points: the beginning, and before and after each of the four compaction events.<br>Each row in the forest plot below shows the difference between one context setting and the setting with nothing, aggregated across the resumption points where that setting exists.
tasks solved
-5<br>+14
turns
-7<br>+5
cost
$+0<br>$+7<br>the conversation<br>without the memory files<br>without the thought content<br>the compaction summary<br>without the memory files<br>the memory files alone<br>+9.9 [+5.9, +13.8]-5.6 [-6.5, -4.8]$+6.33 [+6.06, +6.60]<br>+8.8 [+4.9, +12.8]-3.9 [-4.9, -3.0]$+7.12 [+6.82, +7.43]<br>+8.2 [+4.2, +12.2]-6.0 [-6.9, -5.2]$+6.00 [+5.70, +6.30]<br>+5.4 [+1.8, +9.0]+3.8 [+3.0, +4.6]$+0.14 [+0.11, +0.17]<br>+8.5 [+5.0, +12.0]+3.2 [+2.4, +4.0]$+0.11 [+0.08, +0.14]<br>+1.7 [-5.3, +8.6]+3.4 [+2.2, +4.6]$+0.11 [+0.07, +0.15]
The same six rows as the figure above, as numbers. Effect of each starting state<br>against a session that boots with nothing, on the same 120 held-out tasks, with 95%<br>confidence intervals.<br>starting statetasks solved, percentage pointsturnscost
the conversation+9.9 [+5.9, +13.8]−5.6 [−6.5, −4.8]+$6.33 [+$6.06, +$6.60]<br>the conversation without the memory files+8.8 [+4.9, +12.8]−3.9 [−4.9,...