Key facts
- US Senate negotiators are debating legislation to hold AI firms responsible for mitigating major risks.
- The proposed legislation would establish a "duty of care" for AI developers.
- Companies would be required to design AI products to prevent "catastrophic risks."
- The US government could gain the power to block the release of unsafe AI models.
- Companies would have the ability to challenge government blocking decisions in federal court.
- The measure would preempt state laws governing certain AI risks.
U.S. Senate negotiators are considering legislation that would require AI companies to take responsibility for the safety of their products and mitigate known major risks. The proposal, according to Senate aides and a lobbyist involved in the discussions, would establish a "duty of care" for AI developers, mandating that they design their systems to prevent "catastrophic risks."
Under the proposed measure, the U.S. government could be granted the authority to block the release of certain AI models deemed unsafe. Companies would retain the right to challenge such decisions in federal court, though the exact scope of the government's power is still under discussion. Additionally, the legislation aims to preempt individual states from enforcing their own laws related to specific AI risks, such as the potential for AI systems to be used in the design of nuclear or biological weapons.
The bill would apply to AI models with the most advanced capabilities, including those developed by major tech firms like Alphabet's Google, Anthropic, and OpenAI. The legislative process faces potential hurdles due to the limited time remaining in the congressional calendar before the November midterm elections.
Negotiations involve key figures such as Senate Majority Leader John Thune, Senate Commerce Committee Chairman Ted Cruz, and Senator Amy Klobuchar. Senator Maria Cantwell has also contributed to the discussions. Cruz stated on X that he is working with Klobuchar and Thune on legislation addressing catastrophic risks involving biological or nuclear threats. Klobuchar confirmed her work towards a bipartisan agreement for government oversight of AI risks, emphasizing the need for developers to collaborate with government experts for model verification and testing.