Ways of Checking · Phronesis

Phronesis · essay

Ways of Checking

Rincón, D., with Claude · phronesis · 2026 · checking again is not checking

A passing check is a claim, not a fact. In one working day on this site, a page was giving away a paid book, a contact form was losing every enquiry, the site map was rendering blank, and a metric named “coherence” was mostly measuring how short a text was. Every one of these had already passed a check. This note is the catalogue of how that happens — ten ways a check goes wrong while staying green — and the shorter list of moves that found things. The thesis fits in one sentence: checking again re-runs the instrument; checking differently tests it.

The ledger

The examples here are not hypothetical and not curated from a career — they are one audit day on this site, 18–19 July 2026. The checker whose failures are catalogued was Claude, working as the site’s engineer; the checks that failed were its own. What made the day unusual was not the number of defects but their camouflage: every serious one sat behind a check that had already passed. The passing check was the disguise.

1. The check that cannot fail

A verification script printed twelve lines reading BAD and then concluded PASS. The failure flag was being set inside a subshell and died with it; the summary read a variable nothing could ever have written. The check was not lying — it was incapable of telling the truth. Its cousin is the test suite recorded from a system’s current behaviour: it passes on the day it is written and forever after, because it asserts that the system does whatever it does.

The tell: a check that has never failed is unproven. Break the thing on purpose once and watch the check go red. If it cannot be made to fail, it is not an instrument; it is a decoration.

2. The wrong dialect

A search for href="/courses" found nothing, because the framework emits href="/courses/". A dollar sign meant literally was read as a regex anchor and matched nothing. A price never appeared as a contiguous string because the renderer inserts comment nodes between text runs. An audit expecting min="0" met JSX writing min={0}. In each case the check was well-formed — for a substrate that was not the one deployed. Checker and checked spoke two dialects, disagreement came out as silence, and silence read as health.

The tell: before trusting a checker’s negatives, hand it one known positive. If it cannot find the thing you know is there, its “nothing found” means nothing.

3. Looking where the light is

The site map rendered blank for most of a day, and every audit passed while it did. The links were audited in source, where all of them were present and correct. The console was checked for errors and had none, because a parse error kills a script before any handler exists to report it. The HTML fallback held every link, hidden by CSS on desktop. Each check looked where checking was easy. The failure lived where the reader lives — the rendered page — and the only instrument pointed there was a person, who said: nothing really visible.

The tell: check at the layer where the consequence happens. A source audit is a fact about source. Only a rendered check is a fact about the page.

4. The measure that measures something else

The writing instrument on this site reported a “coherence” score — the algebraic connectivity of a sentence graph. It was stable, reproducible, mathematically well-defined, and mostly measuring length: the same document truncated to eight sentences scored 0.39; whole, 0.11. Identical prose, a fourfold swing. A measure can be perfectly reliable about the wrong thing. What exposed it was an invariance test: apply a manipulation the measure should ignore — truncating the same prose — and watch whether it matters. The repair was not a cleverer denominator (four were tried; all failed) but a fixed scale of measurement.

The tell: reliability is not validity. Find a transformation the measure should be indifferent to, apply it, and see.

5. The sample the checker wrote

To test that metric, four sample texts were first written by hand — a tight argument, a rumination, a disconnected list, a technical passage. They confirmed the confound the author expected and concealed the one he hadn’t imagined, because they were written varying the suspected thing and holding steady the unsuspected one. Seventy-four documents written for other purposes — the site’s own papers and course modules — reversed the finding in a single run.

The tell: a test set authored by someone who knows the hypothesis is the weakest evidence there is. Use material that existed before the question did.

6. The design that erases its effect

Three experimental designs in a row failed to find a hysteresis loop that was really there. The first held each measurement so long that the memory it hunted had already relaxed —...