AI is approving our pull requests: Here's how we made it safe - The Intercom Blog

Skip to main content

At Intercom, shipping is our heartbeat. We push code to production hundreds of times a day.

Engineers, engineering managers, designers, and PMs all contribute to this, safely. The average time from merging code to it running in production is 12 minutes.

We’ve long held a belief that might sound counterintuitive: speed is not the enemy of safety. It’s a prerequisite for it. Accumulating code creates risk. Shipping small batches minimizes it. The faster you ship, the smaller each change is, and the easier it is to catch problems, and roll back when something goes wrong as the context is still fresh in your head.

Today, over 93% of our pull requests (PRs) across our two main codebases are Agent-driven. And over 19% are auto-approved with no human reviewer in the loop.

This post is about that second number, and why we think it makes us safer. Most people hear "AI is approving our pull requests" and think that’s reckless. We think the data tells a different story.

The 2x effort

Last year, our CTO Darragh Curran set an explicit goal: double the productivity of our entire R&D organization within 12 months. Because the faster we can build and ship, the faster our customers get the capabilities they need.

Nine months later, we did it. The results have been significant across the board, but the number that matters most for this post: downtime from breaking code changes dropped 35%, even as our deployments doubled. Shipping faster made us safer. As we modernize how we build and ship software, we systematically surface bottlenecks and tackle them. One of the biggest we’ve found? PR review.

The PR review problem

Humans simply don’t have the time or mental capacity to properly review the volume of AI-generated code we’re now producing.

When an AI Agent can produce a working implementation in minutes, waiting hours or days for a human to review it is an impedance mismatch. The production line is moving faster than the quality gate can keep up.

When that happens, one of two things follows: either the queue backs up and velocity drops, or, more dangerously, humans start rubber-stamping. Glancing at a diff, skimming the description, clicking approve.

Some companies are drifting into this failure mode silently. We chose to confront it head-on and built a rigorous solution.

PR review, done properly, is a genuinely complicated process. A good reviewer assesses the quality of the problem statement based on the PR description. They confirm the changeset actually matches the stated intent. They review the code against best practices, look for logical issues, apply their personal product context to validate the changes make sense, and check for performance issues, safety concerns, and more. No single human reviewer realistically has experience to properly cover every one of these dimensions on every PR, especially not under time pressure. And as we’ll see in the data, the baseline we’ve been comparing against, human review, was weaker than most of us assumed.

So we asked ourselves: what if we could do better?

How our PR review Agent works

Our PR review Agent doesn’t treat code review as a single task. It decomposes it into separate sub-jobs, each handled by an independent sub-Agent. One assesses the quality of the problem description. Another checks whether the diff actually aligns with the stated intent. Another reviews for safety concerns. Another checks for logical correctness. Another reviews against best practices and known anti-patterns. And so on.

The result is that every PR is reviewed as if a dozen of our most tenured and knowledgeable engineers were all looking at it simultaneously, each bringing their own specialist lens. In the past, getting that breadth of review on a single PR was impossible. Now it’s the default.

A human reviewer typically focuses on the actual code changes, the diff. Our Agent goes deeper. It traces execution paths, following the implications of a change through the codebase. This is something humans rarely had time to do, even when they wanted to.

A real example:

While testing our new PR review Agent on a set of historical PRs, we found it flagging a one-line text copy change as incorrect. On the surface, it looked completely harmless, just a text update. We assumed it was a mistake, but it wasn’t. Our Agent caught that the new copy contradicted an existing validation mechanism elsewhere in the codebase. No human reviewer would have realistically found this unless they happened to have written that validation code very recently. Our Agent catches this kind of thing consistently, every time, because it’s always tracing execution.

The review isn’t generic either. It’s grounded in Intercom-specific guidance that our engineers have built and continue to refine, encoding the same context, standards, and product knowledge they’d apply if they were reviewing the PR themselves....