U.S. Senate negotiators are considering legislation that would establish a "duty of care" for AI developers, requiring them to design products to prevent catastrophic risks. The proposal would grant the government power to block the release of unsafe AI models, with companies able to challenge such decisions in federal court. The measure would also preempt state laws governing certain AI risks.
The proposed legislation could significantly impact the development and deployment of advanced AI systems by imposing new safety requirements and regulatory oversight on tech companies, potentially influencing innovation and market access.
U.S. Senate negotiators are considering legislation that would require AI companies to take responsibility for the safety of their products and mitigate known major risks. The proposal, according to Senate aides and a lobbyist involved in the discussions, would establish a "duty of care" for AI developers, mandating that they design their systems to prevent "catastrophic risks."
Under the proposed measure, the U.S. government could be granted the authority to block the release of certain AI models deemed unsafe. Companies would retain the right to challenge such decisions in federal court, though the exact scope of the government's power is still under discussion. Additionally, the legislation aims to preempt individual states from enforcing their own laws related to specific AI risks, such as the potential for AI systems to be used in the design of nuclear or biological weapons.
The bill would apply to AI models with the most advanced capabilities, including those developed by major tech firms like Alphabet's Google, Anthropic, and OpenAI. The legislative process faces potential hurdles due to the limited time remaining in the congressional calendar before the November midterm elections.
Negotiations involve key figures such as Senate Majority Leader John Thune, Senate Commerce Committee Chairman Ted Cruz, and Senator Amy Klobuchar. Senator Maria Cantwell has also contributed to the discussions. Cruz stated on X that he is working with Klobuchar and Thune on legislation addressing catastrophic risks involving biological or nuclear threats. Klobuchar confirmed her work towards a bipartisan agreement for government oversight of AI risks, emphasizing the need for developers to collaborate with government experts for model verification and testing.