I ran Sonnet 5 vs. Opus 4.8 head to head on 24 tasks to see what's different

bisonbear1 pts0 comments

Sonnet 5 vs Opus 4.8: how they behave and when to use each — Stet

Sonnet 5 vs Opus 4.8: how they behave and when to use each<br>July 8, 2026 · Updated July 9, 2026<br>Sonnet 5 is a confusing model. 5 > 4.8, but Opus > Sonnet. When should we use Sonnet 5? In this post, we used a private eval to test Sonnet 5 vs Opus 4.8 across 24 tasks from two open source repos and inspected the behavioral differences to answer when to reach for each model.

TL;DR: on these tasks, Sonnet turned higher reasoning effort into more checking, longer trajectories, and patches the LLM judge scored clearer and more intentional. Opus's activity stayed flatter through high reasoning effort, while the judge leaned toward simpler, more robust, more minimal diffs. Neither profile is universally "better", but the failure modes and working styles shift.

The kicker is price, and output tokens, which changes dramatically with reasoning effort. Sonnet cost 0.62x Opus at low and 0.81x at medium; high and xhigh were effectively tied; at max Sonnet cost 1.37x Opus.

The short answer

If your constraint is…<br>Route I would try first<br>Why, on this slice

Best cost/performance default<br>Opus high<br>Opus's equivalence was already near the top of its curve, while cost rose sharply beyond high.

Ambiguous or verification-heavy work<br>Sonnet xhigh<br>Sonnet's equivalence peaked at xhigh, where 7 of 8 craft dimensions also leaned Sonnet.

Minimal diffs and scope control<br>Opus high<br>Opus high had lower footprint risk on 18 of 22 shared tasks and stayed near its performance plateau.

Routine work where high is unnecessary<br>Opus medium<br>Opus was already near its plateau at medium; low is not the general-value route.

A safe default at max<br>Neither<br>The extra spend did not support a broad default for either model.

This is a routing policy from one local slice, a demonstration of the power of running evals on your own code, not a universal model hierarchy like DeepSWE or FrontierCode.

How we ran it

24 real tasks from two open-source repos: graphql-go-tools (Go) and sqlparser-rs (Rust), each derived from a PR that the maintainers actually merged. Both Sonnet and Opus ran every task at five reasoning efforts (low, medium, high, xhigh, max) in Claude Code, with one run per arm. One GPT-5.4 pointwise judge graded equivalence to the merged fix and eight craft dimensions. Costs are cache-aware geometric means per task.

The cost ladder

cost per task, geometric mean<br>The cheaper model flips with the dial<br>Ratio of Sonnet 5's cost to Opus 4.8's, geometric mean per task, at each reasoning setting. Left of the 1.0x line, Sonnet is the cheaper arm; right of it, Opus is. Low and medium land clearly on Sonnet's side; high and xhigh hug parity; max and the Sonnet-xhigh / Opus-high routing pair land clearly on Opus's side.

Sonnet cheaperEffectively tiedOpus cheaper<br>low

0.62x<br>Sonnet cheaper

medium

0.81x<br>Sonnet cheaper

high

1.17x<br>effectively tied

xhigh

1.01x<br>effectively tied

max

1.37x<br>Opus cheaper

xhigh vs high<br>Sonnet xhigh vs Opus high — the routing pair

1.81x<br>Opus cheaper

Sonnet geomean per task<br>$0.98 (low) → $8.77 (max)<br>9.0x span

Opus geomean per task<br>$1.58 (low) → $6.41 (max)<br>~4x span

Sonnet's dial spans 9.0x, from $0.98 geomean per task at low to $8.77 at max; Opus's dial is flatter, about 4x from $1.58 to $6.41.

The quality trade

One pointwise judge scored eight craft dimensions on every patch pair: clarity, simplicity, coherence, intentionality, robustness, instruction adherence, scope discipline, and diff minimality.

Most individual margins are small, and the eight dimensions are not independent. I read this as a behavioral fingerprint, not a leaderboard. The signal is that the same judge saw the same directional trade across nearly every effort setting. Sonnet's patches were more often judged clear and intentional; Opus's were more often judged simple, robust, and minimal. We can use this to learn more about how the models performed on these tasks, but not to claim that either model writes better code in general.

Here's what leans consistently, and what it means:

Clarity leans Sonnet at all six comparison points. Strongest signal: high effort Sonnet takes 20 of 24 tasks with 2 ties; at xhigh 18 of 21. If reviewer comprehension is the bottleneck, this is the clearest case for Sonnet, especially at high and xhigh.

Intentionality leans Sonnet at all six points. That suggests Sonnet's changes may be easier for a reviewer to connect to the task.

Diff minimality leans Opus at all six points. Strongest: low effort Opus takes 21 of 24 tasks with zero ties; Sonnet xhigh versus Opus high is 17 of 22. This is the case for Opus when patch surface is policy, and it lines up with the deterministic footprint read.

Simplicity. The graders lean Opus on simplicity at every effort level except xhigh, where it tilts Sonnet's way, 11 tasks to 9. The judge more often read Opus as the less complicated fix; xhigh is where Sonnet closes that gap.

Robustness leans Opus at five of six....

sonnet opus high xhigh tasks task

Related Articles