Versioning the Harness Itself. The harness is code. The team executes… | by Ian Johnson | Jun, 2026 | MediumSitemapOpen in appSign up<br>Sign in
Medium Logo
Get app<br>Write
Search
Sign up<br>Sign in
Versioning the Harness Itself
Ian Johnson
6 min read·<br>Just now
Listen
Share
The harness is code. The team executes it the way the team executes code: in production, on real tasks, with stakes. Code gets versioning, change review, and migration discipline. The harness usually has none of that, and the absence costs us.<br>What versioning actually means<br>I do not mean a version number in the file header. That would be theater. The harness lives in git already; commit hashes are the version.<br>I mean the practice around changes: how a change is proposed, how it is reviewed, how it is communicated, how it is rolled out, and how it is rolled back. The discipline that surrounds any other change to shared code. The harness deserves the same discipline because it has the same blast radius. A rule change reaches every engineer the next time they start a session.<br>The harness is infrastructure. Treating it casually because it is a markdown file is the category error.<br>The change review nobody runs<br>Most teams I have talked to do not review harness changes the way they review code. The change lands in a PR, someone glances at the diff, the rule reads sensibly in isolation, the PR is approved. The conflict check, the scope check, the who does this affect check — none of it happens.<br>The minimum review I run now, on any non-trivial harness change:<br>Does this conflict with an existing rule. A grep against the harness for the same concepts, the same file paths, the same workflows. If the new rule overlaps with an old one, the change is to merge or replace, not to add.<br>Is the scope right. Project root, subdirectory, path-glob. A rule about the API code should not be in the project root if it never applies elsewhere.<br>What workflows is this change going to affect. Not “what tasks.” What recurring workflows. A rule that says always do X in the import flow affects every engineer who touches the import flow next week. The change is a coordination event.<br>Is this reversible. Some rule changes are: delete the rule, the agent goes back to whatever it did before. Some are not. The rule taught the team a habit, and the habit will persist after the rule is gone, possibly carrying the rule’s mistake with it.<br>The review takes ten minutes. It catches most of the failures before they ship.<br>When a rule change is actually a migration<br>The harness sometimes needs more than a rule change. It needs a migration. The structure shifts; a section moves; the conventions for how rules are written change. Those changes break every contributor’s mental model at once, the same way a directory restructure breaks every IDE’s open tabs at once.<br>The teams that handle this well treat it the same way they handle a code migration: announced ahead of time, deployed on a Monday, with a written migration note in the PR. The note says what changed, why, what to update if you have in-flight work, and who to ask if something looks broken.<br>The teams that handle this poorly merge the migration on a Friday afternoon and answer DMs about it for the next week.<br>The cost of the announcement is fifteen minutes of writing. The cost of not announcing is hours of fragmented confusion across the team, plus the slow erosion of trust in the harness that comes from being surprised by it.<br>Rolling out without breaking everyone<br>The rollout discipline matches the change’s blast radius.<br>Get Ian Johnson’s stories in your inbox
Join Medium for free to get updates from this writer.
Subscribe
Subscribe
Remember me for faster sign in
Additive change. A new rule that does not conflict with anything, scoped to a path that already has a CLAUDE.md, applies to new work only. Merge it. The team notices the next time the agent acts in that path.<br>Change to an active workflow. Post in the team channel before merging, name the workflow, name the change, give people a chance to push back or flag in-flight work. Wait a day if anyone has a branch in that workflow. The cost is one day of delay; the benefit is that the rule lands without breaking active work.<br>Structural change. Write a migration note, schedule it for a Monday morning, batch other harness changes into the same window if they overlap. The team gets one event to update their mental model rather than five spread over the week.<br>A rule about commit messages is not the same blast radius as a rule about how the API layer handles errors. Treating them with the same care is overkill on one and insufficient on the other.<br>The rollback that earns its place<br>The rollback is easy because the harness is in git. Revert the commit, the agent reads the old rule on the next session, the workflow recovers.<br>The rollback that is not easy is the one where the rule taught the team a behavior that survived the rule. A rule that said name files like X sat in the harness for...