Can I trust the numbers? · Zaki Hasan

The most common question I've been asked while building and selling Honeycomb is some version of: "How can I trust the numbers?"

It's the right question. If you're using an LLM to produce output that has to be not just plausible but verifiably correct - the kind people make consequential decisions on - then trust isn't a nice-to-have, it's the whole product. This piece is about how I've come to think about earning it: the infrastructure that sits around the model and decides what the system is actually willing to stand behind. The examples come from Honeycomb, but the approach isn't specific to it, and I think it generalises to any domain where extracted data has to be good enough to act on.

The question behind the question

When someone asks whether they can trust the numbers, they're usually not asking about model benchmarks or which provider you use. They're asking what happens between extraction and the moment a value shows up in a report, a dashboard or a calculation. Can they see where it came from? Can they tell how sure the system actually is? What happens when something looks wrong?

Those are reasonable questions, and they're the ones that matter commercially, because they're the basis on which someone makes a decision. They're also questions most LLM extraction pipelines aren't built to answer well. The default pattern is simple: the model extracts a value, the model assigns a confidence score, the value gets stored and used. That confidence score feels like a safety mechanism, but in practice it's the equivalent of the model grading its own homework.

Honeycomb started out extracting from documents that feed financial and technical analysis, so this was never abstract for us. If extracted values are going to flow into valuations, comparisons and reports that people act on, you need a clearer account of what was extracted, what was checked, and what the system is prepared to stand behind. That shaped how we designed the whole pipeline.

Verification comes first

The first layer I'd put in place - before almost anything else - is independent verification that runs on every extracted field before it's stored or used anywhere downstream.

Every extracted field arrives with a source quote: the passage the model says it took the value from. Before that value goes anywhere, the system checks whether the quote - or at least the value itself - actually appears in the source document. This is deterministic text matching, not another model call: normalised matching, with fuzzy matching for quotes that come back slightly paraphrased. No second model grading the first.

It's worth running that check against two independent representations of the same document: the model's own transcription, and a separate raw text extraction taken straight from the file. That's deliberate, because the two fail in different ways. The model produces fluent, coherent text but can misread a table or quietly smooth over an awkward layout; raw extraction preserves what's literally on the page but can miss things the model captured correctly. Checking against both gives better coverage than either alone.

The result isn't a single pass/fail. Each field is tagged with one of a few outcomes: the quote matched verbatim; the value is present but the quoted passage didn't match; neither was found; or there was nothing to check it against. Those are genuinely different situations, and downstream handling should treat them differently - a paraphrased source is not the same as a value that appears nowhere in the document, and collapsing the two into one flag quietly degrades both your confidence policy and your review experience. Whatever the outcome, that tag stays with the field permanently.

The real value here isn't only catching bad extractions, though it does that. It's making verification a recorded property of the data. When someone later asks why a particular value was treated cautiously, or why it was flagged for review, you can point to a specific check that ran at extraction time and exactly what it returned. That matters enormously the moment you're past the demo and into production.

Confidence should be a policy, not a model output

Verification gives you something concrete to base confidence on. So rather than trusting the model's self-reported score, apply a confidence policy: a set of rules, defined explicitly in code, for how confident the system is allowed to be given how a value was obtained and what verification found.

The model still reports its own confidence, but that only moves the score within a band the policy sets. The shape that's worked for us: a value that was directly quoted and verified is allowed into a high band - but never treated as ground truth. The same value, with a quote that couldn't be found anywhere in the document, is forced into a low band regardless of how confident the model claimed to be. A value the model inferred rather than quoted sits lower than...