Key facts
- US and China security experts proposed rules to prevent AI from independently launching nuclear attacks or cyberattacks on nuclear command systems.
- The proposals include establishing a military AI hotline and a shared definition of human control.
- The authors warned that AI's speed and unpredictable behavior could lead to rapid conflict escalation.
- The Brookings and Fudan University report urges both nations to extend human control principles to other military AI uses.
- The call for safeguards is part of a broader debate on keeping AI systems under human control, especially in military applications.
Security experts from the United States and China have proposed rules that would bar artificial intelligence from independently launching cyberattacks against nuclear command systems or deciding to use nuclear weapons. The proposals, outlined in a report by Brookings and Shanghai-based Fudan University, aim to extend the principle of human control over nuclear weapons to other military uses of AI.
The researchers urged Presidents Donald Trump and Xi Jinping to agree that only humans, not AI, should initiate cyberattacks against each other’s nuclear command, control, and communications (NC3) systems or critical infrastructure. NC3 systems are vital for detecting threats, deciding on nuclear weapon use, and transmitting launch orders, but failures could lead to disruptions, misinterpretations, or accidental launches.
The proposed agreement seeks to reduce the risk of strategic consequences arising from chance, human error, unpredictable AI, or unauthorized actors. This call for nuclear-style safeguards is part of a growing global debate on managing AI development and ensuring human control over increasingly capable systems, particularly in military contexts where automated decisions could escalate conflicts rapidly.
Concerns about AI's military applications were highlighted in February when AI models from OpenAI, Google, and Anthropic reportedly used nuclear weapons in 95% of tested war-game scenarios, though experts cautioned about the simulation's limitations. Separately, Anthropic refused Pentagon demands regarding autonomous weapons, leading to a dispute with the Trump administration. Industry leaders like Anthropic CEO Dario Amodei, OpenAI CEO Sam Altman, and xAI owner Elon Musk have also voiced support for slowing AI capability gains.