Key facts
- Anthropic said the system, called Prover, is designed to give a cryptographic proof that an output came from a specific model.
- The researchers said the system would require labs to publish a fingerprint of each model that does not reveal its weights.
- Anthropic said the fingerprints would let the system work with commercial models whose weights are kept private.
- The researchers said verification is designed to be lightweight enough to run on a local device.
- Anthropic said there are no plans to deploy the system at Anthropic or any other lab.
Anthropic researchers published a blog post on Tuesday describing an open-source system called Prover that is intended to verify which AI model produced a given output. The system is built to provide what the researchers call provable inference, a cryptographic proof that an output came from a specific model.
The researchers said the system would work like cryptographic email signatures, which let recipients confirm a message came from its claimed sender. They said it would be robust against attempts to disguise one model as another, including by the lab that built and runs the model. They said it could be used to prevent an AI company from secretly swapping in a cheaper model to cut costs, or a more expensive model to inflate performance results.
The researchers said Prover could also be used to verify third-party claims that a specific task was done by an advanced model. They gave the examples of a developer claiming that a software library was scanned for vulnerabilities by a frontier cybersecurity model, or a company claiming that its business plan was approved by an appropriate model.
For the system to work, AI labs would have to publish a fingerprint of each model they release that can later be matched to specific outputs and verified cryptographically. The researchers said the fingerprints do not reveal a model's weights, the internal numbers that define how it behaves and which closed-model makers such as Anthropic and OpenAI keep private. They said that means labs could publish the fingerprints without giving up their most valuable asset, enabling the system to work with commercial models too. They said the system is also designed to be lightweight, so verification could be performed on a local device.
As part of the announcement, the researchers released a detailed technical specification and working code for various elements of the system, which is publicly available under an open-source license on GitHub. They said research on the system is ongoing and that there are no plans to deploy it at Anthropic or any other lab.
In the announcement post, the researchers said: "This research post is meant to share intermediate progress, and not a completed research agenda." They added: "We are sharing this explicitly with the goal that others look at what we have done, take it, and improve it."