Key facts
- Anthropic and OpenAI are negotiating publication rights with independent AI evaluators.
- Companies fear that findings from embedded evaluations could reveal trade secrets.
- Measures discussed to prevent leaks include lab employee observation and restricting simultaneous work at different labs.
- Experts note that what labs deem intellectual property can overlap with risk assessment information.
- Anthropic stated that in current conversations, outside reviewers control their methods, conclusions, and what to publish.
- OpenAI said it redacts confidential information and evaluators can disclose major redactions.
Leading artificial intelligence companies Anthropic and OpenAI are engaged in sensitive negotiations with independent evaluators regarding the publication of their findings. The core of the dispute centers on the AI labs' concerns that evaluators, particularly those working closely within the companies' walls, could inadvertently leak proprietary trade secrets to rivals. This tension arises as these companies pursue massive valuations and potential IPOs, balancing the need for transparency in AI safety with the protection of their intellectual property.
Anthropic and OpenAI have reportedly sought significant control over what external evaluators can publish. Measures under discussion to mitigate intellectual-property leaks include having company employees observe embedded evaluation teams and preventing evaluators from working with competing labs simultaneously. However, experts note that the definition of crucial intellectual property can overlap with information vital for assessing AI model risks, leading to potential disagreements and redactions.
While Anthropic and OpenAI have publicly stated support for evaluator independence, the specifics of their agreements are still being ironed out. Anthropic clarified that in ongoing discussions, it has emphasized that outside reviewers control their methods and conclusions, but reserves the right to comment on and approve reports from new firms. OpenAI stated it does not seek full approval rights but redacts confidential information, allowing evaluators to disclose if major redactions occurred. Contracts already in use can bar firms from publishing confidential information without company consent.
The outcome of these negotiations is seen as critical for establishing how powerful AI technologies, which some creators warn are advancing beyond human control, will be scrutinized. Without agreed-upon standards and legislation for independent reviews, there is a risk that companies could dictate the terms of examination, potentially compromising the integrity of safety assessments.