Anthropic has released FlashProve, an open-source research prototype dedicated to provable inference. This system, based on a zkSNARK (a cryptographic proof that allows demonstrating a claim without revealing its details), makes it possible to prove that a response comes from a fixed model, without revealing anything about its weights.
- Anthropic releases FlashProve, an open-source prototype that proves a response comes from a fixed model without revealing its weights.
- The provider publishes a commitment offline, and the GPU generates a proof for each query online.
- Proofs cost about 72 times more GPU time than vLLM per batch, and verification takes 17 seconds.
- The system is limited to 1,024 tokens per query, greedy decoding, and MoE models.
The problem targeted primarily concerns developers. Today, a user querying a model cannot verify which model actually produced the response. A routing service could replace the requested model with a cheaper one, and an attacker could modify the token stream (the units of text processed by the model). Judging the quality of a response is not enough, according to the source, to detect these substitutions.
Operation and performance
FlashProve involves two actors: the prover, i.e., the model provider, and the verifier, i.e., the user. The system operates in two phases. Offline, the provider publishes a commitment, a short fingerprint that binds an exact set of weights. This step takes about 39 minutes and covers about 16 million queries. Online, the GPU produces for each query a proof that the committed model indeed generated those tokens.
Anthropic claims that this setup offers post-quantum security and indicates that it belongs to the same family as the validity proofs used by layer 2 networks like Starknet. According to the repository’s performance guide, Anthropic measured the system on the public weights of Llama 4 Maverick, a 400B parameter mixture-of-experts (MoE) model (an architecture that only activates a portion of the parameters per token), with 4-bit precision, on a single H100, an Nvidia GPU.
For a batch of 24 queries, the announced throughput is 111.23 tokens per second. A single query only reaches 17.41 tokens per second. The proof size is 18.78 MB, and its verification takes 17 seconds on 24 threads. Compared to vLLM, an inference system running Maverick in FP8 on 8 H100s, the proof costs about 72 times more GPU time per batch, and about 94 times at the maximum throughput tested by vLLM.
Security and limitations
Cryptographic code being fragile, Anthropic says it has formalized the protocol in the Lean language, to prove its completeness, validity, and zero-knowledge disclosure. The README indicates that a cheating provider would be detected, except with a probability of about q/2^124 + 2^-101. FlashProve includes two verifiers: a fast C++ verifier, intended for daily use, but which has no formal guarantee, and a slower Lean verifier, whose acceptance is limited, according to Anthropic, to what the validity theorem admits.
The system also has constraints. It is currently limited to 1,024 tokens per query, greedy decoding, and MoE models. Anthropic also indicates that its team used Claude to optimize the system. The code is published under the Apache 2.0 license on the FlashProve GitHub repository.
