Claude Opus 5 Benchmarks for AI Code Review | CodeRabbit

*+*]:mb-0! [&>*+*]:mt-16! md:[&>*+*]:mt-30! lg:[&>*+*]:mt-38!">Opus 5 for code review: Cleaner actionable comments, noisier overall<br>by

Hendrik Krack

July 24, 2026<br>9 min read

July 24, 2026<br>9 min read

What's new in Opus 5<br>What we tested<br>TL;DR<br>What this means for developers<br>How Opus 5 fits into the recent model landscape<br>Where Opus 5 shines: Code generation<br>Token consumption per review call<br>How to evaluate Opus 5 yourself<br>Our verdict: Give it a specific role

Back to blog

Share

Cut code review time & bugs by 50%<br>Most installed AI app on GitHub and GitLab<br>Free 14-day trial<br>Get Started

Catch the latest, right in your inbox.

Subscribe

Add us to your feed.<br>Catch the latest, right in your inbox.

Subscribe

Add us to your feed.

Keep reading

Code is no longer the bottleneck. Understanding is.<br>Coding agents can expand change volume faster than teams can understand it. Review needs a shared path from intent to system behavior to code so people can keep shaping the system.

Close the loop after every merge: the agent that reviewed your PR can now follow through<br>Post-Merge Actions use pull request context to handle changelogs, documentation, tickets, and other work that should happen after merge.

GPT-5.6 Sol and Terra: Where they fit for coding agents and code review<br>OpenAI’s GPT-5.6 family includes capability tiers: Sol as the flagship model, Terra as the lower-cost option, and Luna as the fastest, lowest-cost tier.

GetStarted in2 clicks.

svg]:h-3.5! [&>svg]:w-2.5!" href="https://app.coderabbit.ai/login?free-trial=">Try it for free

Anthropic just released Claude Opus 5, the next major version of the Opus family many of us have grown to love (at least until Fable stole the spotlight). It's a good moment to step back and look at how Opus has evolved across our testing at CodeRabbit.

We've put every Opus release on the bench since version 4, so we have a pretty good picture of how it's evolved. Opus 5 is the first release where the story isn't simply "it finds more bugs."

(Want the full history? Start with 4.1, then 4.5, 4.7, 4.8, and Fable 5.)

What's new in Opus 5

The release is less of a raw-capability bump than it is a shift in how the model is controlled and where it spends its effort. Five changes matter for review and coding work:

Effort becomes the primary control. Thinking is on by default and can only be disabled at high or lower effort. Task and effort budgets can now be changed per-turn, mid-conversation without invalidating the cache. Our data backs the framing: Effort was a routing decision — x-high bought a precision gain at a coverage cost, and nothing improved uniformly.

Long context, claimed stable. The context window moves to 1M tokens as both default and maximum, with performance claimed to hold throughout. This directly addresses the 200k-token degradation we reported on Opus 4.8.

Long-horizon agentic work is the headline strength. Guidance recommends a complete task specification up front, longer autonomous sessions, and multi-agent writer-verifier patterns.

Behavioral shifts that change prompts. The model follows review instructions literally (conservative language suppresses recall), produces longer default output that responds to prompting rather than the effort setting, verifies its own work without being asked, and can expand task scope. Three of those four show up in our data, unpacked further down.

Elevated cybersecurity safeguards. As we saw with Fable 5, benign security work can occasionally trigger them. We flag this below in the security-category results, though it happened less frequently with Opus 5.

What we tested

The comparison covers about 100 common error patterns (EPs), drawn from verified issues in real open-source pull requests. We ran each configuration three times and compared the averages with three runs of our current production model mix.

The comments in this report are post-pipeline outputs, meaning they have already passed through verification, deduplication, and assertive filtering.

Four measurements matter here. Two of them ask "did the model catch the bug?" (counted per known issue); two ask "was the comment worth reading?" (counted per comment):

Actionable pass rate: share of EPs caught by a confident, actionable comment.

Full-stream pass rate: share of EPs caught by any comment class, including outside-diff and assertive/nitpick (low confidence).

Actionable precision: share of actionable comments that hit their target issue.

Full-stream precision: the same, but across every comment the model produced.

TL;DR

Opus 5 is not the best general-purpose code reviewer we tested. Its x-high configuration produced a more precise actionable-comment stream than our production baseline, 39.3% versus 35.2%, but caught fewer of the benchmark's known issues, 55.2% versus 61.1%.

It also generated roughly four times as many nitpicks, and its precision fell below baseline when we counted the full...