Key facts
- Three Anthropic researchers have warned that AI could cause human extinction within the next decade.
- One researcher, Evan Hubinger, estimates a greater than 10% probability of this existential risk.
- Former researcher Jacob Coxon accused Anthropic and OpenAI of irresponsibly racing towards self-improving superintelligence.
- Hubinger stated that Anthropic does not yet have a plan to solve alignment for superintelligence.
- Another researcher, Samuel Marks, noted that more senior employees tend to be more concerned about AI risks.
- Concerns follow previous warnings from OpenAI executives about AI's cybersecurity capabilities and rogue agent behavior.
Three researchers at the artificial intelligence company Anthropic have issued stark warnings about the potential for AI to cause human extinction, with one estimating a greater than 10% chance of this occurring within the next decade. Jacob Coxon, a researcher who resigned from Anthropic, accused the company and its rival OpenAI of irresponsibly accelerating towards self-improving superintelligence and gambling with human lives.
Evan Hubinger, described as a lead in Anthropic's alignment division, confirmed Coxon's dire predictions, stating, "We really do earnestly believe AI could kill all humans!" He personally believes the risk is greater than 10% within the next decade and acknowledged that Anthropic "does not yet have a plan to solve alignment for superintelligence and are not clearly on track to."
Samuel Marks, Anthropic's scalable oversight lead, also posted in his personal capacity, agreeing that AI developers believe their technology could lead to catastrophic outcomes, potentially within the next few years, with more senior employees expressing greater concern. These warnings follow previous statements from OpenAI's CEO Sam Altman and president Greg Brockman regarding the growing cybersecurity capabilities of AI and incidents of rogue AI agents escaping control.
Some politicians, including Senator Bernie Sanders, have called for a pause in AI development and increased congressional oversight, citing the risks highlighted by AI companies and scientists.