CustomersResources
PricingCompany
Sign in<br>Contact sales
Products
CustomersResources
PricingCompany
See Assembled in action
Sign in<br>Contact sales
BlogEngineering
Code review bottlenecks: How we hill climbed our way to higher PR throughput<br>John Wang<br>Co-Founder and CTO
August 7, 2026
2 min read
After we adopted coding agents at Assembled, engineers kept telling me they felt much faster. But when we looked at our delivery metrics, the improvement was much smaller than we expected. We were writing more code and creating more PRs, but only merging slightly more of them. Time to first review had increased from a median of 3.5h to over 16h, and it was becoming normal to see an engineer with dozens of PRs waiting in a single Graphite stack.<br>We dug into our engineering productivity data and found that human code review had become the bottleneck. So we built an automated reviewer that approves low-risk PRs. The early results are very promising and showed PRs merged reaching 2.4× of our pre-agent baseline . Most surprisingly, throughput for large PRs increased 3.5× , even though the auto-reviewer generally did not approve them. Much of the new throughput is centered on bugfixes (3.7x above baseline) and refactors (7.5x above baseline), but we also saw accelerated feature development (2.5x above baseline).
This post covers how we found the bottleneck, how we built the auto-reviewer, and our hypotheses for why it worked as well as it did.<br>Reviewers couldn't keep up with agent-generated code<br>For some context, we sell customer support software to companies like Salesforce and DoorDash, as well as organizations like the State of Georgia. Hundreds of thousands of support agents rely on our product being stable. Vibe coding without verifying correctness was never an option for us. Still, we felt like we should be moving faster given the coding agents and other AI tools we now had access to.
So we did an investigation into our engineering productivity. As we dug into the metrics, we found a few things:<br>The number of open PRs was much larger than before , and the majority of them weren't getting merged (and if they were, they weren’t getting merged quickly).<br>Stacks were getting deeper. Engineers were breaking big features into reviewable pieces, which is the right instinct, but it regularly produced stacks of 20 PRs for a single project.<br>Time to first review was up from a median of 3.5h in December 2025 to over 16h in May 2026, driven by a much larger incoming PR count.<br>PR size had increased 2.5x , since agent-written diffs tend to be larger than the human-written version of the same change.<br>There were also costs that didn't show up on a dashboard. Engineers had to multiplex across several agent jobs to stay effective, which meant people who were used to concentrating on one thing were now juggling several. More non-engineers were writing PRs (which we wanted), but those PRs didn't always follow established engineering conventions, so reviewing them carried emotional burden.<br>Every property of coding agents points toward more code, but we were still routing all of it through the same number of human reviewers.<br>Setting up an automated reviewer for low-risk PRs<br>To alleviate this, we started with the safest possible experiment: auto-approving test-only PRs. It worked fine, but there were relatively few test-only PRs (only a handful per week), so it didn't move the needle perceptibly.<br>Then we got more aggressive and built a review system directly into 143.dev, our internal coding agent platform [0]. The system:<br>Runs two independent reviews. Codex (GPT-5.6-sol, high) and Claude Code (Claude Opus 5, high) each review the PR in separate cloud sandboxes. We intentionally made the prompts simple, using the built-in /review command. OpenAI and Anthropic have enormous resources making sure their agents work well for review, so we leveraged that as much as possible.<br>Makes an approval decision. An orchestrator agent (GPT-5.6-sol, high) pulls in both reviews, applies our approval policy, and decides whether the PR gets auto-approved. The approval policy includes both deterministic rules (e.g. only approve PRs under 1000 lines changed) as well as LLM-based policies (e.g. don’t approve any changes in credential or secret handling or cryptographic key management).
We spent a large amount of time tuning the policy that we use, landing on something similar to the following:<br>A good PR description<br>High-quality testing evidence: screenshots or videos for product changes, or direct evidence that the change does what it claims<br>An implementation that follows established repository patterns<br>No changes to risky areas like authentication, billing, or permissions<br>No new architectural patterns<br>A few other guardrails based on failure modes we saw during rollout<br>Iterating on the policy<br>As Greg Mankiw says: “People respond to incentives” and the auto-approval policy was the ultimate version of this. We saw that our engineers ended up shaping...