Attestable Blog - Proving LLMs at Scale
Get in touch
[ AI integrity ]<br>Proving LLMs at Scale<br>How Attestable moved zero-knowledge inference beyond toy models, with production-scale LLM proofs, compact proof artifacts, sub-second verification, and mostly floating-point execution.
Get in touch
Table of contents
Table of Contents
Related Resource<br>Model Security
From Verifiability to Model-Weight Security
AI workloads are easy to claim and hard to prove.<br>A model provider can say that a particular model processed a particular input. A cloud can say it served the requested weights and architechture. An AI lab can say its system followed an approved policy. But unless the relying party can inspect and rerun the entire computation, those claims ultimately depend on trust.<br>Zero-knowledge proofs can replace that trust with mathematics. A proof can establish that a committed model produced a specific output from a specific input, without revealing the model’s weights or other private information.<br>The problem has always been performance.<br>Proof generation that works for a small classifier is not automatically useful for a production LLM. Frontier-scale inference involves billions of parameters, long contexts, floating-point arithmetic, and enormous matrix multiplications. A theoretically sound proof system is irrelevant if producing the proof takes hours, days, or more compute than the world can afford.<br>Today, we are publishing the first results from Attestable’s prover for production-scale transformer inference.<br>On a single NVIDIA H100, our current system reaches:
Our proving throughput, proof size and verification time for various models. All benchmarked on a single H100, with context size 4K and batch size 4.Across all successful benchmark configurations, proof sizes range from 4.35 to 7.92 MiB , and verification takes 157 to 648 milliseconds on our reference CPU verifier.<br>These are alpha results. There is still substantial optimization ahead of us. But they put zero-knowledge proofs for LLM inference into a fundamentally different performance regime.<br>This is no longer ZKML for toy models. For comparison, the natural open-source alternative, ezkl, claimed in the end of 2023 a proving time of 16 minutes for a 1M parameter model on a CPU (x70,000 smaller model, and about x50,000 slower!).<br>What exactly are we proving?<br>For a model (F) with committed weights (W), input (x), random seed (r), and output (y), Attestable generates a proof of a statement of the form:<br>y = F(W, x,r)<br>The proof binds the execution to a cryptographic commitment to the model weights. It can also bind the input, output, model configuration, decoding parameters, and other policy-relevant values.<br>The proofs are short, and verification is fast and easy , and does not depend on how large the model or how long the context is.<br>A verifier can then check that:<br>the committed model was used<br>the claimed input was processed<br>the claimed output resulted from that execution<br>the required execution policy and random seed were followed<br>The verifier does not need the model weights and does not rerun the model .<br>You can think of this as a digital signature created by the computation itself , rather than by an organization holding a signing key. Change the model, input, output, or proven policy, and verification fails.<br>Our prover operates alongside the inference process. Once a sequence has been decoded, the system produces one proof covering the complete benchmarked execution. The proof can then be checked independently by a standalone CPU verifier.<br>Our security and limitations<br>All of our results are based only on the security of cryptographic hash functions, and thus we are post-quantum secure . We are not relying on public-key cryptography at all, and uphold 100 bits of security - meaning, a malicious attacker would need to compute 2100 values before they can bypass our solution.<br>In our current version, we have two temporary limitations:<br>We support up to 16K tokens in the context window.<br>Though we prove all non-linear operations in floting points, we quantize all matrix multiplications to integers.<br>We believe both limitations will be lifted in the not-too-far future.<br>Gemma 4 31B as a case study<br>Gemma 31B is a useful test of whether a proof system is ready for real LLM workloads, as a dense medium-sized model.<br>Performance across different configurations<br>Proving throuhgput depends on the batch size and context length. Our current Gemma prover reaches 53 tokens per second for a batch-one 16K sequence. At batch four, it reaches 77 tokens per second across four 4K sequences.
Prover throughput for Gemma 4 31B on a single H100What our quantization does to Gemma?<br>Performance is only useful if the proven model remains useful. Compared to other ZKML systems that only use integers, we use floating points for almost all of the operations during inference. Our only current limitation is dynamically quantizing the linear operations to 8-bit integers.<br>We evaluated the...