A judge ran the same prompt three times and got three different answers - Johannes Stillig

A judge ran the same prompt three times and got three different answers

5 Aug 2026<br>19 min

I'm Johannes. I build an investigations engine where every finding has to trace back to the source bytes it came from, which turned out to be an awkward promise once a language model started helping produce the findings.

In a trust accounting case in New York, an expert witness told the court he had used an AI assistant to cross-check his damages calculation. The judge asked him what he had typed into it. He could not remember. The judge asked what sources it had used. He could not say.

So the judge typed the question in himself, on a court computer, and got $949,070.97. He ran it again on a second court computer and got $948,209.63. A third returned a little over $951,000.

Three court-issued machines, one question, three answers. The Surrogate's Court wrote that while the variations were not large, the fact that there were variations at all called the reliability of the output into question, and held that counsel has an affirmative duty to disclose the use of AI before such evidence is admitted.

That is Matter of Weber, decided in October 2024. The expert was not caught fabricating anything. He was caught being unable to say what he asked or what came back.

I have spent the last month making sure that cannot happen to my product. What I built works, and it proves less than I originally claimed it did, and the gap between those two things is worth writing down.

Reproducibility is the wrong thing to want first

The instinct, when you notice this problem, is to reach for reproducibility. Pin the parameters, set a seed, get the same answer twice.

You cannot, and the reason is more interesting than floating point.

Thinking Machines Lab published the clearest account of it. The usual explanation, that concurrent floating point addition is non-associative, turns out to be wrong: run the same matrix multiplication on the same data repeatedly and you get bitwise identical results every time. The actual culprit is that your output depends on the batch you happened to be in, and the batch depends on who else was hitting the server at that moment. Their measurement: a thousand completions of one prompt at temperature zero produced eighty unique completions . The first 102 tokens were identical every time. Then they diverged.

From your point of view, the other users of the API are not an input. They are a property of the weather.

This is fixable, and it has shipped. Batch invariant kernels give bit identical output, and both vLLM and SGLang now expose a deterministic mode. The cost is real, somewhere between a quarter and two thirds of your throughput depending on whose measurement you take, and there is a nastier property underneath: determinism is contagious. One deterministic request joining a batch of ten ordinary ones dropped total server throughput from 931 tokens per second to 415. It is not a per-request option you can quietly buy for yourself.

And none of it is available to you on a hosted API. The Anthropic Messages API has no seed. On current frontier Claude models you cannot even set temperature: a non-default value returns a 400. Bedrock's Converse inferenceConfig has four knobs and none of them is a seed. OpenAI, the one provider that offered a seed and a backend fingerprint, has marked both deprecated: true in its OpenAPI spec and shipped neither on the Responses API.

So the industry's one reproducibility affordance is being withdrawn while everyone talks about AI governance.

Which is fine, because reproducibility was the wrong thing to want first. Logging is attestation, not reproduction. It proves what you sent and what came back. It does not prove the model would say it again. Those are different properties, and only one of them is available to you today.

The expert in Weber did not lose because his answer was irreproducible. He lost because he had no record.

Nobody has specified what to record

Here is what surprised me most. I went looking for the standard that says what to capture about a single model call, expecting to find several and have to pick.

There is not one.

"Traceability" appears zero times in the NIST AI Risk Management Framework. So does "record". The Generative AI Profile says "provenance" sixty-seven times, and every one of them is about content provenance or training data, because the problem NIST is solving there is deepfakes, not defensibility. Every AI bill-of-materials format describes the model rather than the call: CycloneDX's inputs and outputs fields hold formats, not values. Sigstore's model signing signs a list of file digests, which tells you the weights were not tampered with and nothing about what they did.

The closest thing to a specification comes from an unexpected direction. OpenTelemetry's GenAI conventions define exactly the attributes you would want, including...