Efficient proofs of language-model inference
How can a client check that a large language model used committed weights to produce an output, while keeping those weights secret and spending much less work than running the model? A succinct cryptographic proof lets the server provide evidence of that computation. The task is to reduce proof time, memory and size while keeping the security assumptions and the exact inference rule explicit.
Why it matters and what counts as progress
Why it matters: A client normally has to trust a remote provider’s statement about which model ran. Efficient proofs can make that computation checkable without sharing the weights. Their guarantee concerns the specified computation; assessing the answer’s factual accuracy is a separate task.
What counts as progress: Build a faster or smaller proof system for a stated model and arithmetic rule; prove its completeness, soundness and privacy under explicit assumptions; or verify that executable code matches the formal protocol. Report hardware, context length, batch size and how many output tokens are actually checked.
How the problem developed
- 2017 · Verifiable neural-network inference [1]
Zahra Ghodsi, Tianyu Gu and Siddharth Garg SafetyNets used interactive proofs to check inference for networks represented as arithmetic circuits, with experiments on small image and speech models. It did not provide model-weight privacy.
- 24 Apr 2024 · Zero-knowledge transformer inference [2]
Haochen Sun, Jason Hongyang Zhang zkLLM introduced lookup and attention proof techniques and reported inference proofs for a 13-billion-parameter model. Its model, hardware and proof scope differ from the later FlashProve experiment.
Claude with Roy Rinberg, Alex Allain and collaborators FlashProve combined cryptographic tools into a protocol for an integer version of Llama 4 Maverick. The authors reported single-H100 benchmarks and Lean protocol-security proofs, while leaving fast-code equivalence and batched zero knowledge unresolved.
Selected milestones in verifiable neural-network inference. Reported timings across different models and hardware are not controlled speedup comparisons.
AI contribution
FlashProve: an open-source library for provable inference
FlashProve combines a hiding weight commitment with a succinct cryptographic proof of integer-model inference for Llama 4 Maverick. For a batch of 24 requests with 1,024-token contexts, the authors report about 223 seconds on one H100 GPU and 24 CPU cores, with one selected token checked per request and an 18.78 MB proof for each request. The approximately 110 tokens/second figure counts processed context tokens; checking every token instead gives about 39.1 checked tokens/second in their experiment.
Sources
Publication history
Daily emails by default, only when there’s an update.