The correction loop is the bottleneck — Kinesthetic<br>We keep finding the same process inside teams building specialized agents for production: a spreadsheet of exported traces from their observability platforms, annotated by a subject matter expert, is converted into a Linear issue/Jira ticket and eventually lands as a barely tested prompt edit by an engineer who wasn’t in any of the conversations about this. That pipeline is where most vertical AI agent improvement actually happens today: a mirror of the adhoc process that works for software development, but does not hold when working with machine learning models.

Here, we’ll discuss why this pipeline exists, why we think it is the biggest constraint on enterprise agent deployment, and what we think replaces it.

AI is not hype, but there is a real gap

AI is not hype. It feels fair to say that coding agents are the definitive proof that agents work. We (and most engineers we talk to) derive real value from Claude Code, Codex, Cursor, etc. in our work. Additionally, companies like Sierra, Clay, and Harvey have all built durable businesses on agents doing domain-specific work outside of software. Recently, Harvey just hit their first $100M+ net new ARR quarter.

Yet there is still a stark difference between the magic of coding agents and the reality of agents performing other domain-specific work. The same companies pushing the boundary of non-coding agents are publishing open benchmarks exposing the gap between what the current models can do in more realistic scenarios. If we look at the Harvey Legal Agent Benchmark leaderboard and the Sierra τ³-Banking leaderboard, we can see that a frontier model like Fable 5 scores 14.2% and 26.8% respectively. The best frontier models pass the large majority of individual rubric criteria on both benchmarks while fully completing only a small fraction of tasks.

This failure shape exposes something interesting. The model is able to utilize the provided domain knowledge well, but it still fails to deliver work that a professional would sign off on. The gap isn’t general knowledge, it’s knowing the procedure to complete a task and understanding the specification across a long horizon without dropping the nuanced requirements that nobody explicitly wrote down for it.

Why coding got there first

Two structure advantages that don’t exist (easily) for other domains:

Coding (and math) exists as the most verifiable-rewarded task – allowing for easy RL env building – which makes synthesis or quality checks over this data so much easier.

The sheer amount of largely trustable, information-dense data – long horizon over repository diffs with PR descriptions, review threads, CI outcomes, etc. – is the type of abundance and quality that you won’t see in other enterprises or use cases.

There’s also a third advantage that gets less attention. On a coding agent, the engineer building the system is the domain expert. There is no translation layer, no game of telephone about how a domain expert would do something. The person who knows what “correct” means is the same person who is changing the harness or developing the RL environment.

None of this holds in auditing, legal, financial, underwriting, or other white collar work. There’s no ready-made open test suites or benchmarks. The correctness is a matter of subjectivity and professional judgement rather than logical execution and observation. This leaves one question: in a domain with no verifiable reward, where does the training signal come from?

Inside most teams, it comes from a person. While agentic judges have become a standard, most of the folks we’ve talked to find that the only reliable source of ground truth is the expert (often in-house) who reviews the agent’s work and annotates what was wrong and why. In these non-verifiable verticals, the SME is the gradient .

Which is why the correction loop is the whole problem

If the expert’s annotations are the learning signal (gradient), then the path from that export to an improved agent is your learning rate. Right now, that path is a game of telephone from the domain experts, PMs, and engineers that creates lossy corrections – often resulting in unstable or incomplete improvements, forcing another cycle of annotation, triaging, and development.

We’ve talked to dozens of teams building agents and have realized that the operational inefficiencies are structural, not cultural, especially because of how nascent the technology and infrastructure is. Engineers don’t want PMs to update prompts because they don’t know how those prompts or knowledge are retrieved/used across the harness, PMs want to double check with SMEs when their engineers make arbitrary decisions about domain knowledge that doesn’t look right, and SMEs are producing an overwhelming amount of feedback that the engineering team simply does not know how to prioritize. A correction gets exported to a spreadsheet, re-encoded as a ticket, half-implemented in one of the...