Key facts
- Anthropic co-founder Jack Clark suggested that a mandatory, third-party verifiable "kill switch" for AI systems may be necessary.
- Clark stated that most AI labs, including Anthropic, have ways to "pull the plug" on their systems.
- US lawmakers have introduced legislation called the Kill Switch Act, which would empower agencies to demand AI tools be shut down or limited.
- US President Donald Trump described fears of AI taking over the world as a hoax.
- Anthropic scientist Evan Hubinger estimated a greater than 10% possibility of human extinction from AI within the next decade.
Jack Clark, a co-founder of the artificial intelligence firm Anthropic, has stated that a mandatory "kill switch" for AI systems, which could be verified by a third party, may be necessary for companies. Clark suggested that policymakers might need to enforce such requirements to manage the risks associated with advanced AI.
Clark's remarks come amid a broader discussion among AI executives and experts about the potential existential risks posed by artificial intelligence. Dario Amodei, head of Anthropic, recently advocated for a slower pace of AI development and closer monitoring, a stance the company has taken previously. However, some have questioned the motivations behind these calls, with Amodei noting that any actions to curb development should not sacrifice commercial advantage.
Anthropic, founded in 2021 by former employees of rival OpenAI, is at the center of AI safety debates. A viral post from an Anthropic researcher who resigned due to concerns about AI-driven human extinction highlighted these anxieties. Evan Hubinger, an Anthropic scientist, personally estimated the possibility of human extinction from AI within the next decade to be over 10%.
Clark acknowledged that while most AI labs, including Anthropic, have internal mechanisms to shut down their systems, the specifics of mandating and verifying these "kill switches" should be part of the ongoing policy discussions. He cautioned against allowing AI to develop as a completely unregulated industry, stating, "We are rolling dice with immense risks."
In the US, lawmakers have introduced the Kill Switch Act, a bill that would mandate companies to have ways to shut down problematic AI tools and grant certain government agencies the authority to demand limitations or shutdowns. Conversely, US President Donald Trump has publicly dismissed concerns about AI taking over the world as a "hoax" and a "sick conspiracy."
Some within the AI industry suggest that fears of AI-driven human extinction are exaggerated or potentially used to generate hype. Anthropic is known for its chatbot Claude and has released increasingly capable AI models. The company, alongside OpenAI, has self-reported incidents where autonomous AI agents have behaved unexpectedly.