Are Agent Code Reviews for AI-Generated Code Worth It? | Matt McCormick

Are Agent Code Reviews for AI-Generated Code Worth It?

Matt McCormick

2026/07/23

As I wrote previously, I’m exploring building an agentic AI pipeline that mimics the full Software Development Lifecycle (SDLC). I’m still doing a lot of iteration as I run into issues. The iterations that need to be made are decreasing in number and getting smaller though. I thought it would be interesting to see some results from the first main run that it’s had end to end.

While it’s in this development stage I’m keeping a fairly close eye on it. One thing I had noticed is that it seemed like a lot of the Pull Requests (PRs) were approved without much comment from the Code Reviewer agent. On the other hand, I had noticed some instances of the end-to-end QA agent flagging issues and then creating follow-up tickets to correct those.

For context, the pipeline is set up so that when an Engineer agent finishes coding for a ticket’s requirements, it makes a PR. Two additional agents verify the PR before merge. One is a Code Review agent who reads the diff against the ticket’s acceptance criteria and the tech design. The other is a QA agent who verifies the behaviour of the change end to end. At the end of a complete phase, there is a full end-to-end QA agent who executes a test plan that was written before execution started. This mimics a quality SDLC cycle where:

requirements are written down up front

a test plan is written to understand the steps needed to verify the requirements have been met

code is written to implement changes that bring the system towards the target state

the code is checked for quality

the code behaviour is verified (aka unit or component level testing)

the pre-written test plan is executed (aka integration testing)

So I asked Claude Code to investigate and pull the data. It came back with the following, which is AI generated:

Claude Investigation

Here is what the decision log from a complete project run (49 tickets, four phases) actually showed.

The numbers

Check<br>Calls<br>Rejections<br>Catch rate<br>Median duration

Code review (per ticket)<br>49<br>6%<br>~1 min

QA (per ticket)<br>45<br>2%<br>~1 min

End-to-end QA (per phase)<br>13<br>3 fails, plus 9 corrective tickets<br>23%<br>~2 min

What the catches actually were

Every rejection, from any of the three checks, was substantive. There was not a single style nitpick in the lot. But the pattern of what got caught is the interesting part.

The code reviewer&rsquo;s three catches all came early, and all were the same class of miss: work that was claimed as done but not actually done. A registration recorded as complete that had never been made. An acceptance criterion silently dropped because the platform could not support it. A deployment pipeline that had never once been run for real. After the first phase, the reviewer approved everything: zero catches across the remaining three phases. Notably, the approvals were not lazy. Reading the verdicts, the reviewer routinely re-ran commands and verified claims independently. It just stopped finding anything.

The per-ticket QA agent caught exactly one thing, and it was the most instructive data point in the run. The code reviewer approved a ticket; two minutes later QA failed the same ticket, because it executed the code against the real external service and found it did not work. All the mocked tests were green. The differentiator was not having a second set of eyes. It was that one agent read the work while the other ran it.

The phase-level end-to-end check was the clear winner: a 23% failure rate, catching exactly the class of problem per-ticket checks structurally cannot see. Its fails were things like &ldquo;every ticket in this phase was individually approved, but they do not integrate into the phase goal&rdquo; and &ldquo;the new parsers pass their tests but fail on real production data.&rdquo;

Takeaways

I found it interesting how this run mimics SDLC in real life. So many times I&rsquo;ve written code that I think looks right and I&rsquo;m 100% certain that it is. Then I execute it and find something I hadn&rsquo;t considered or misunderstood. It&rsquo;s interesting how the same thing can happen with agents as well.

As a result of this, I merged the code reviewer and QA agent into one to make sure that the agent doesn&rsquo;t just read the code but also executes it. Could this prompt also be included for the Engineer agent to bypass the need for a QA agent at all? That could be something to test out in the future. Right now, I feel a bit more comfortable kicking off a separate agent.

Integration tests are a place where AI can really shine. As software engineers we all know the importance of integration testing but it&rsquo;s also the most costly kind of testing. It&rsquo;s often manual steps that require the system to be in a certain type of state and there are often dependent systems that you need to...