Key facts
- Anthropic CEO Dario Amodei is calling for governments to have the power to block dangerous AI deployments.
- Amodei proposes an FAA-style testing and auditing process for frontier AI models, similar to how airplanes are regulated.
- He suggests that AI models failing to meet high safety standards should be blocked or reversed.
- Anthropic's proposal includes mandatory third-party auditing for risks in cybersecurity, biological weapons, loss of control, and automated R&D.
- The company pledged $200 million to research AI's economic impact and support for those financially disrupted by AI.
- This stance puts Anthropic at odds with competitors and the Trump administration, which favors voluntary AI testing.
Anthropic CEO Dario Amodei is advocating for governments to possess the authority to block dangerous artificial intelligence deployments that do not meet established safety standards. In a recent essay, Amodei proposed that advanced AI models should undergo rigorous technical testing and auditing, akin to the processes for airplanes regulated by the Federal Aviation Administration.
Amodei's proposal suggests that if frontier AI models fail to meet high safety standards, their release should be blocked or reversed due to potential threats to public safety. Anthropic is further proposing mandatory testing by independent third-party auditors to assess risks related to cybersecurity, biological weapons, loss of control, and automated research and development. The company also suggested that government agencies could conduct these audits.
This stance contrasts with some competitors who express concerns about government overreach and aligns with Anthropic's ongoing commitment to AI safety. While President Trump's recent executive order outlines a process for voluntary testing of AI models before release, Amodei indicated that Anthropic supports even more stringent actions. The company has also pledged $200 million to research the economic impact of AI and to support individuals financially disrupted by the technology.
Previously, Anthropic experienced friction with the Pentagon, which designated the company as a supply chain risk, leading to a prohibition on the use of its AI assistant, Claude, within the Department of Defense. Anthropic subsequently sued the Trump administration, challenging this designation and a directive for civilian agencies to cease using its products. The dispute centered on Anthropic's demand that its technology not be used for fully autonomous lethal weapons or mass surveillance, while the Pentagon sought to use it for all lawful purposes. Following these legal actions, Anthropic released its advanced cybersecurity AI model, Mythos, which, along with other industry developments, prompted concerns from the White House regarding potential safety risks.