Engineer Two - Miguel Conner
Miguel Conner
SubscribeSign in
Engineer Two<br>the art of designing cleverer checks for your loop.
Miguel Conner<br>Jul 03, 2026
Share
Outer Sunset, San Francisco, California. May 2026.<br>I was inspired to start experimenting again with AI coding after reading a quote from the CTO of Speak, Andrew Hsu, about how his team thinks about building with AI. Their framework contrasts two hypothetical engineers:<br>Engineer one is a software engineer who uses Claude Code or Codex for about 90% of what they do. Fundamentally, though, they’re using it to move faster, but nothing has actually changed in their mindset for how they build.<br>Engineer two , on the other hand, actually realizes that their job has changed, from working on the implementation of the feature to building the environment and systems to make an AI agent work better. They’re building that feedback loop and adding capabilities to the agenda so that it can do all of the work, including verifying the feature that it just built.1
This framing is both clear and yet invites questions. How do these feedback loops actually work? What makes feedback good?<br>Well here is my attempt at answering those questions. I’m adhering as closely as possible to the Engineer Two philosophy while building a personal project from scratch. My plan: fine-tune a language model for language simplification. My criteria for judging this project: fun had, quality of result, quality of code, and time taken.<br>This did not go as planned. I knew going in that the key would be building good verifiers, but I found the practical implementation to be tricky. In fact, I rewrote this post 3 times because I kept having to redesign my approach. But I trained my model and learned a few tricks about designing good feedback. If you will allow me: it’s a good model, sir.<br>Thanks for reading! Subscribe for free to receive new posts and support my work.
Subscribe
A Bottleneck
Engineers One and Two are distinguished by the bottleneck of human validation. Engineer One sets a small task for an agent, waits for it to finish the task, checks how it handled the task, then gives the agent another task. Going from this to Engineer Two is really the process of outsourcing this verification step and giving the agent a roadmap for how to proceed, so that it can quickly iterate unencumbered.
Fig. 1: Verifying things is tedious. And the models still need some amount of it, as this meme suggests.<br>I’m not claiming that checking the code manually is bad. But if you’ve used an LLM before you know they will try different, often wrong, things. If you have to manually verify every one of these ideas and tell it to try something new, then the process is not fun and also takes a really long time. The game is setting up the agent in such a way that the engineer reviews the code only when it’s mostly finished and working correctly. The new bottleneck is setting up verification.<br>Verification
Traditional ML models update weights based on loss functions. AI agents update code based on verifiers.2<br>An agent that can verify the quality of the work it has done so far, will go much further than one that simply assumes the work it has done so far is good. Verification here can operate at a few different levels, and it’s worth thinking a bit about the difficulty of certain verification processes.<br>Asymmetric verifiability is a concept discussed in the context of training LLMs, but it is also extremely relevant to AI coding. It describes a case where it is much easier to check than to solve a problem. For example: solving a crossword is much harder than checking an already filled-out crossword. Sometimes you need an answer key for these kinds of problems, but other times, you can skip the answer key all together or even design your own. And as frontier models improve in ability, the scope of asymmetrically verifiable things potentially increases as well.<br>The challenge is that LLMs can be a bit wishy-washy, often changing opinions readily. And unfortunately, the best signals for agents need to be consistent, accurate, and meaningful.<br>Low-Level Verification<br>We can apply the principle of asymmetric verifiability to writing code if we consider that writing a test for a function is usually much simpler than writing the code to do the function.<br>One strategy for improving code reliability with feedback loops is a technique called red-green Test-Driven Development (TDD). The red-green describes the fact that the agent should first write a test before writing the code it will need to test. The first time the agent runs the test it should of course fail (red!), and then after it has written the actual code it runs the tests again and they will ideally pass (green!). This actually works pretty well for delivering working code since it creates extremely clear feedback loops for the agents that help them through difficult coding tasks.<br>But while red-green TDD can verify that code executes correctly, not adding further...