The Model Writes, the Judge Measures: Anatomy of an LLM judge | XInfer.AI
MenuDocumentation
Listen to full article~21 min
Listen to AudioSpotlight~3 min
The Model Writes, the Judge Measures: Anatomy of an LLM judge
By Sam Wen, CEO & Founder of XInfer.AI — July 23, 2026
Here is the kind of sentence that should keep a team up at night:
"At this clarity grade, you won't see a thing with the naked eye."
Our assistant produced it while explaining a diamond. The grade it cites is real — SI1, printed on<br>the stone's lab report. The promise attached to it is invented. A clarity grade is assigned under<br>10× magnification; it does not resolve what a naked eye will see. Two SI1 stones can look entirely<br>different — one clean at arm's length, the other with a visible speck right in the middle.<br>The model took a true fact and manufactured its consequence, because that is what a helpful<br>sentence sounds like.
Notice what the sentence is not. It is not a banned word. It is not a hallucinated stone. It<br>contains no wrong number. It breaks no rule you could have written down in advance. It is simply<br>fluent — and fluency is the problem. Fluent and true are different variables, and a language model<br>optimizes the first one.
We build AI assistants for exactly this kind of product. The one in this essay explains diamonds:<br>a short, two-voice spoken conversation about one stone — a host asks the questions a shopper<br>would ask, a gemologist answers — generated from that stone's evidence. A diamond is the<br>definition of a high-consideration purchase: thousands of dollars, bought rarely, chosen under<br>uncertainty. And it has a property that makes it an unforgiving test bed for generative AI —<br>nearly every claim about a diamond can be checked against its certificate from GIA or IGI, the<br>independent laboratories that grade the world's diamonds. Overselling is not just tacky here. It<br>is checkable.
In an earlier essay we argued that the language model should conduct the conversation while a<br>deterministic core makes every decision. This essay is the same worldview applied to quality. The<br>model writes. It does not get to grade.
At one generation you can catch a sentence like the one above by reading it. Across a catalogue of<br>stones you cannot. So before we made the writer better, we built an instrument: an independent LLM<br>judge that reads every generated claim against the evidence and returns a verdict we can count.<br>This essay is about how that judge is built, how we keep the judge itself honest, and what changed<br>once we had it — which is that the arguments stopped.
Part I — Three gates, and who owns which rule
The assistant explains one diamond at a time, from that diamond's evidence: the lab certificate,<br>the price, the retailer's published analysis of the stone. One rule governs everything: every<br>claim about this stone must trace to supplied evidence. If the evidence cannot answer — is it<br>eye-clean, meaning no marks visible to the naked eye? — the assistant says so, out loud: the<br>honest answer is a maybe, and it will not promise more. Then it points the shopper to a human gemologist.
Enforcing that rule is not one mechanism. It is three, layered, and they differ in kind.
The first gate prevents. It runs in code, inside the pipeline. When the assistant generates a<br>follow-up question about a stone, the question must name the evidence path it would be answered<br>from. A question that names no available path is rejected — by code, not by prompt hope. This is<br>what stops the system from asking "how does this price compare to the market?" about a stone<br>whose data has no comparable set. The model never gets the chance to improvise an answer, because<br>the question never survives.
The second gate catches. A deterministic validator runs over every finished script: a<br>hard-fail lexicon of the words the assistant must never say — amazing, must-have,<br>investment, the manufactured urgency, the "trust me." Word-boundary regexes, a few dozen tests,<br>no judgment involved. If a banned phrase appears, the script fails. Cheap, fast, and absolute.
The third gate judges. What remains after prevention and regex is the dangerous class: the<br>plausible-but-unsupported claim. "You won't see a thing with the naked eye" passes both gates. No<br>rule names it in advance, because it is not a phrase — it is a relationship between a sentence and<br>a lab report. Checking it requires reading both. That is a job for a reader, and at scale the<br>reader has to be a model.
One detail here taught us more than we expected: the boundary between the gates is itself a<br>design decision. When we extended the banned-word list from the domain experts' style guide, the<br>unambiguous terms went into the regex gate. But the context-dependent ones deliberately did not —<br>and the best example is flawless. In most products, "flawless" is empty praise. In diamonds it<br>is literally a grade: FL, the top of the GIA clarity scale. "It is not flawless" is a factual<br>statement about an SI1 stone, not a...