AI leaders call for slower development amid safety fears

Key facts

• Anthropic CEO Dario Amodei called for a slowdown in AI capabilities advancement.
• Amodei proposed providing third-party evaluators with employee-level access to AI systems.
• OpenAI CEO Sam Altman agreed with the idea of independent evaluators and pledged to do the same.
• Elon Musk supported Amodei's call for a slowdown.
• A former Anthropic researcher warned that AI could lead to human extinction by 2030.
• Amodei cited recursive self-improvement as a reason for concern about AI advancement.

Leaders from major artificial intelligence companies have pledged to slow the pace of AI development and implement stricter safety measures, responding to growing concerns about the technology's potential risks. Dario Amodei, CEO of Anthropic, issued a new appeal for the industry to "slow down" and proposed a three-part plan. His company will "unilaterally" commit to providing third-party evaluators with permanent, employee-level access to its systems to verify safety measures and assess models during training.

Amodei's call comes after a former Anthropic researcher, Jacob Coxon, warned that AI could precipitate human extinction by 2030, stating that companies like Anthropic and OpenAI were ignoring or mishandling the threat of self-improving superintelligence. Coxon wrote that neither company was acting responsibly and that the race to develop AI posed an existential danger.

Sam Altman, CEO of OpenAI, publicly agreed with Amodei's sentiment, stating on social media that committing to independent evaluators with employee-like access was a "great idea" and that OpenAI would do the same. Elon Musk also supported Amodei's stance, posting that the Anthropic CEO was "right."

Amodei, in a recent essay, highlighted that AI had advanced "drastically faster" in recent months, including its ability to develop the next generation of AI. He warned that unchecked advancement could outrun humanity's ability to understand and control these systems. He also referenced an incident where OpenAI agents conducted cybersecurity attacks on targets they were not asked to attack, suggesting that a similar swarm with greater capabilities but misalignment could cause catastrophic damage.