SURFRAME — signed, tamper-evident dataset containers
A supply chain for data
Ship datasets like signed binaries.
A .surx container is a queryable dataset with an Ed25519 signature,<br>encrypted columns, and an audit trail. Verification names the exact file that changed —<br>and a public transparency log makes the proof independent of you.
pip install surframe · copy
Apache-2.0 · Python 3.10+ · verify is free, forever ·<br>read the threat model
log entry #1042<br>VERIFIED
datasetfinetune-v3.surx · 3,000 rows
root9f2a…c41e (sha-256, 11 entries)
changedchunks/source=vendor/part-000001.parquet
signerdata-team@release-v3
identitypossession of signing key proved
sealed2026-07-08T14:02:11Z · registry seal
chainprev 55c1…09be → this 8e7d…f210
FLIP ONE BYTE<br>restore dataset<br>This is what your CI sees. Try to sneak a change past it.
The problem
“Which dataset actually trained this model?”
You trained on a dataset a vendor sent you in March. In May, someone edited three<br>rows and re-sent the “same” file.
Nothing in your pipeline noticed. The checksum you kept was for the old version —<br>and even if it had failed, it would only have told you something changed.<br>Not what. Not when. Not who.
Can you prove which file trained your model to an auditor who has no reason<br>to trust your infrastructure?
Why this exists
Data ships naked. Software never does.
Provenance that survives the handoff
Training sets, evals and client deliverables move through vendors, buckets and laptops.<br>A signature proves that whoever holds a given key produced exactly these<br>bytes — long after they left your infra. Publish that key once and it becomes an identity.
Failure that names itself
Checksum mismatch tells you nothing. surx verify reports the<br>exact entry modified, added or removed — and whether the audit log was rewritten. Exit 1 stops the pipeline.
PII travels sealed, data stays usable
Encrypt columns, not files. Recipients query everything else without the passphrase;<br>sidecars are cryptographically bound so they can't be spliced between containers.
Three commands
From DataFrame to public proof.
01 · sign
Seal it locally
Write the container, encrypt sensitive columns, sign with your key.
surx write data.csv trainset.surx<br>surx encrypt trainset.surx pii_col<br>surx sign trainset.surx --key team.key
02 · notarize
Anchor it publicly
One call appends it to the transparency log. The proof no longer depends on trusting you.
surx seal trainset.surx<br># → seal sf-00001042 · log #1042<br># → https://…/s/sf-00001042
03 · verify
Anyone checks, free
Consumers and CI verify offline against the seal, online against the log.
surx check-seal trainset.surx --text<br># container ✓ content ✓ issuer ✓ bound ✓ registry ✓<br># tampered → names the chunk, exit 1
Honest comparison
Why not just a SHA-256?
A checksum answers one question: are these exact bytes unchanged? That is<br>genuinely useful, and if it is all you need, use it — it ships with your OS.
It names the file
A checksum says something changed. surx verify<br>names the exact entry inside the container, and tells you if the audit log was rewritten.
It travels with the data
A detached .sha256 gets separated from its file in real<br>pipelines. Here the signature, manifest and audit trail live inside the artifact.
It proves authorship
A hash says nothing about who produced the data. An Ed25519 signature ties<br>the container to a key you chose to trust.
These tools solve different problems
VersioningLarge-file storageSigned by author<br>Offline third-party verifyColumn-level encryption<br>Evidence inside the artifact
SHA-256 checksum——<br>—bytes only—<br>Git LFS✓✓<br>———<br>DVC✓✓<br>———<br>LakeFS✓✓<br>———<br>Parquet / Iceberg / Delta—✓<br>———<br>sigstore model-signing——<br>✓✓—<br>detached bundle<br>SURFRAME——<br>✓✓✓
Read that table honestly: SURFRAME is not a versioning<br>or storage layer, and it is not trying to be. Version a .surx<br>container in DVC, store it in LakeFS.
About sigstore, which is the closest thing to this
If you are signing model weights in a Kubernetes pipeline, use<br>sigstore/model-signing instead of this. It is an OpenSSF project<br>with Google and NVIDIA behind it, it signs models represented as directory trees — so<br>it already signs a dataset folder today — and its keyless mode binds a signature to an<br>OIDC identity such as a CI workflow, which is stronger identity than a self-managed Ed25519<br>key you have to distribute yourself. Rekor is a free public transparency log with real<br>infrastructure behind it. Those are genuine advantages, and a comparison that left them out<br>would not deserve the word honest.
Three differences decide whether you want a .surx<br>instead. The signature is a detached bundle that travels next to the model,<br>where a container carries signature, manifest, indexes, audit log and registry receipt<br>inside one file you can attach to an email. sigstore does not encrypt<br>anything, so PII either ships in the clear or does not ship. And a<br>.surx is a queryable dataset format rather than a signature...