Key facts
- Anthropic urged U.S. Congress to enact rigorous federal AI safety laws.
- The company called for mandatory independent safety tests for powerful AI models.
- Anthropic opposes blocking state AI regulations without a federal law in place.
- The Trump administration is considering pre-release vetting for advanced AI systems.
- Discussions involve formal reviews and government-led testing for high-risk AI.
- Anthropic also advised modernizing unemployment benefit payment technology for potential AI-driven layoffs.
- Anthropic suggested empowering private organizations to audit AI models across cybersecurity, biological weapons, loss of control, and automated R&D risks.
Anthropic, an artificial intelligence company, is urging the U.S. Congress to establish a comprehensive federal regulatory framework for AI, warning of catastrophic risks associated with the technology's rapid development. The company specifically called for mandatory independent safety tests for the most capable AI models and advised against blocking state-level AI regulations unless a robust federal law is enacted. Amodei also suggested empowering private organizations to audit models across four main risk areas: cybersecurity, biological weapons, loss of control of AI systems, and automated R&D.
These calls come as the Trump administration is reportedly in early discussions about potentially vetting advanced AI models before their public release. These conversations are focusing on systems that could facilitate cyberattacks, such as those identifying and exploiting software vulnerabilities. Options being considered include formal pre-release review processes and government-led testing for higher-risk systems. Donald Trump's earlier executive order revoked previous AI safety initiatives, but current discussions suggest certain security risks may warrant a more interventionist approach.
Anthropic's introduction of its Mythos model, which has demonstrated significant capability in finding software vulnerabilities, has been a catalyst for these discussions. The company has been briefing government agencies like the Cybersecurity and Infrastructure Security Agency and the Commerce Department on the implications of such advanced models. OpenAI has developed a comparable model and is also engaging with select companies.
Additionally, Anthropic urged Congress and states to modernize the technology used to pay unemployment benefits, citing fears that AI could lead to widespread job losses. Senator Bernie Sanders last week proposed a 50% tax on stock offerings of top AI companies, while Donald Trump floated an unspecified "partnership" with leading AI firms to disburse dividends to Americans. Anthropic has also considered wealth-sharing frameworks, publishing a paper in October suggesting a national sovereign wealth fund with AI investments could help distribute revenue generated by the technology.