Opus 4.6 Was the Last Coding Model That Changed How I Work
Awaiting Input
SubscribeSign in
Opus 4.6 Was the Last Coding Model That Changed How I Work<br>Models have kept getting better. My workflow has not changed nearly as much.
Brandon Sovran<br>Aug 08, 2026
Share
It seems like every month a new coding model lands with yet another chart showing how “groundbreaking” it is, and cue the fear mongering about how this new model is going to kill software engineering for good. Don’t get me wrong, I believe the improvements. I use these newer models myself daily. But the only time since the initial launch of copilots in 2023 I felt the ground “breaking” was when Opus 4.6 got released this past February. 1<br>I became comfortable delegating a large (yet scoped) task and letting the agent run. I could give a goal, let it explore the codebase, answer a couple of its questions, and then let it do its thing. Cursor’s Plan Mode better illustrates this loop; you ask, the agent formulates a plan, then executes it.<br>Did my productivity scale with Coding Benchmarks?
If you read the paragraph above, you know my hunch was no. But I wanted to check whether that was just a feeling, so I went back through my work from September 2025 through July 2026 and built a rough output measure. I assigned delivered work a simple weight based on how substantive it was: routine = 0.5, medium = 3, high = 8, flagship = 13. The inputs were JIRA tickets closed, PRs merged, incident follow-ups, design docs and larger projects shipped.
Substantive engineering output, normalized to the Sep–Jan average 2<br>My substantive output jumps from a Sep–Jan baseline of 100 to about 163 in February. March is 163, April 168 and July 163. The specific numbers are hand-wavy at best, but the shape is what interests me: spike when the actually workflow changed, then no obvious second step-change as newer models arrived month after month.<br>Correlation doesn’t always equal causation, but I’m trying to quantify how the job felt. To me using Opus 4.6 felt like a shift, and this corroborates that story for me.
As LLMs continue to make gains on coding benchmarks, the benchmark I care about (delivering shareholder value), hasn’t gone up. I suspect until we see a new paradigm shift in how we can interact with these agent teams, our productivity gains will continue to stagnate.<br>Coding Benchmarks aren’t the whole job
Coding benchmarks show real progress, sure. The models are better at solving some isolated software tasks, and those gains aren’t totally fake either. The scoring can be quite deceptive if you want to actually get real engineering output though. METR is unusually explicit about this: its time-horizon tasks are designed to be self-contained and well-specified, and it warns against reading it as a high-context engineer’s actual work. 3<br>HiL-Bench is a rare benchmark that gets closer to the failure mode I do care about. It starts with tasks a model can solve, removes information that is required to solve them, and gives the agent an ask_human() tool. Now the agent has to notice that something important is missing instead of confidently filling in the blank itself. 4
Scale AI HIL-BENCH judgment gap<br>Seeing this for the first time helped me back up what is was feeling. The model can be capable of implementing the right solution once it has the facts, while still being bad at recognizing that the facts it has are insufficient. What gets me is that the code often looks fine. It’s just based on the wrong assumption.<br>Caveat, newer models are improving here too. On Scale’s separate combined live leaderboard, Opus 5 is at 57% Pass@3 versus 38% for Opus 4.6. 5 That is real progress.<br>Yet it still only captures one piece of the puzzle. It can test whether an agent notices a missing piece of information. It doesn’t change the way we actually perform the work. Even if Opus 5 asks me more questions, I’m still there working with the agent in real time, and when it says it’s done you are most likely going to have to correct it anyways, just like you would have with Opus 4.6.<br>I tried giving the agents a “manager”
I’ve been pushing on this more aggressively using Gas Town 6, where a “Mayor” coordinates a crew of coding agents. My goal is to stay out of the low-level loop, I don’t have time to do all this micro-management of my agents. I give higher-level tasks to the Mayor, use longer planning sessions, keep persistent project directives, and let the crew distribute the work.<br>It works decently well. Individual tasks can disappear into the crew and come back mostly finished.<br>The problem appears over longer stretches of work: drift<br>An agent finds a niche engineering problem and starts solving it. The solution is often reasonable in isolation. But when I come back and use the product or look at the architecture as a whole, I realize we have spent too much effort on something that barely matters, added complexity I did not want, or slowly moved away from the original product...