Show HN: ARF – a record format for AI evaluation runs, with reproducible digests

ARF - a record format for evaluation runs | KorvoAtlas Record Format · v0.1 ARF - a record format for evaluation runs So a score can be traced back to the question, model, inputs and judgement that produced it. Records canonicalize to identical bytes, so two implementations reach the same SHA-256 digest. Korvo Atlas is the reference implementation. View on GitHub Stability v0.1 is unstable and may change without a migration path before v1.0. Treat published shapes as provisional. Disagree with a shape, or have a case it cannot express? Open an issue - v0.1 is the right time to break it.

What breaks today Most evaluation pipelines end in a CSV of scores. Weeks later a row says 0.82, and nobody can say which prompt version, which model build, or which judging step produced it. The cost lands on whoever has to defend the number: the engineer asked why a regression appeared, the reviewer asked what the score covered, the auditor asked to see the inputs. The scores survive. The context that made them mean anything does not.

What ARF is Ten JSON record types and one hashing rule. ARF is not a new format - it is the format Korvo Atlas already implements, written down so the same bytes and digests can be checked outside the product. Validation is fully offline: no network calls, and schema $id values are never dereferenced. Those identifiers (namespace path korvo.xyz/arf/ns/v0.1) name types - they are not fetch targets, and nothing serves them yet.

A record, in full Every ARF record is a plain JSON object. This is a real Claim, copied from claim.ndjson. Claimexamples/claim.ndjsonCopy{ "id": "6251dbd4-a1f2-4b8b-9cd0-594a46a2f5c3", "statement": "Training language models on their own synthetic outputs leads to progressive degradation of output quality, a phenomenon termed \"model collapse\".", "confidence": "high", "sourceIds": ["bafb465b-47be-45fd-b94f-ea6599153e4f"], "questionId": "7ebb02e3-c3db-4010-a03c-13d7e7ca4df8", "status": "published", "tags": ["model collapse", "synthetic data"], "origin": "human", "reviewStatus": "unreviewed", "createdBy": "Alice Chen", "createdAt": "2026-03-25T20:17:56.727Z", "updatedAt": "2026-03-25T20:17:56.727Z" statementThe assertion itself - one claim per record, never a paragraph containing several. sourceIdsThe inputs this claim rests on. At least one is required, and order is significant: reordering it changes the digest. questionIdThe question this claim answers. Claims point at questions, not the reverse. confidenceHow confident the author is - one of low, medium, high, verified. It is a stated position, not a computed score; engine verdicts live on Evaluation instead. Records are stored one per line as .ndjson. Unset means the key is absent - ARF never writes null to mean “no value”.

How records reference each other Every link is a UUID held in a field on the pointing record. Read X --field--> Y as “X.field holds the id of Y”. Reference directionCopyClaim --questionId----> Question Claim --sourceIds-----> Source Artifact --claimIds------> Claim Artifact --sourceIds-----> Source Evaluation --claimIds------> Claim Challenge --claimId-------> Claim Challenge --challengerId--> Validator Endorsement --claimId-------> Claim Endorsement --validatorId---> Validator Revision --artifactId----> Artifact ChainRecord --artifactId----> Artifact

ArtifactBundle { artifact, question, claims[], sources[], publishedBy, publishedAt } this whole object is the value that gets hashed An ArtifactBundle is the publish-time envelope: one artifact, its question, and the claims and sources it used, collected into a single JSON object. That object is the thing that gets hashed.

The ten record types Grouped by role. Required fields are exactly those marked required in each schema. Inputs What the run was given.

Questionexamples/question.ndjson The question under evaluation. required: id, text, status, createdAt

Sourceexamples/source.ndjson An input that can be cited. required: id, type, title, createdAt

Artifactexamples/artifact.ndjson A written output assembled from claims and sources. required: id, title, type, body, claimIds, sourceIds, createdAt

Assertions What was asserted, and what an engine concluded.

Claimexamples/claim.ndjson One assertion, plus the sources it rests on. required: id, statement, confidence, sourceIds, createdAt

Evaluationschemas/evaluation.json An engine result about claims. The engine payload is stored verbatim. required: id, claimIds, engine, protocolVersion, payload, createdAt

Review Who checked it, and what they said.

Validatorexamples/validator.ndjson A human or agent that reviews claims. required: id, name, type, createdAt

Challengeexamples/challenge.ndjson A recorded objection to a claim. required: id, claimId, challengerId, reason, createdAt

Endorsementexamples/endorsement.ndjson Recorded support for a claim, carrying a weight. required: id, claimId, validatorId, weight, createdAt

Lineage How an artifact changed, and what was...

Show HN: ARF – a record format for AI evaluation runs, with reproducible digests

Related Articles

(no title)

Scientists reverse brain aging, with a nasal spray

NHS apologises and admits Palantir have access to identifiable patient data

AI has torched the market for junior programmers

Al Vigier: Canada's AI strategy shouldn't include secret Palantir bills