Key facts
- AI researcher Jacob Coxon resigned from Anthropic due to concerns about AI's existential threat.
- An Anthropic alignment lead expressed belief that AI could kill all humans, estimating a greater than 10% chance within the next decade.
- The warnings come amid increased AI capabilities and following a hack of an OpenAI internal model.
- Some observers question if these warnings are a tactic by companies preparing for IPOs.
- Anthropic CEO Dario Amodei has since published a plan for more cautious AI development.
Leading AI researchers have raised alarms about the potential for artificial intelligence to pose an existential threat to humanity, with some warning of catastrophe "any day now." The current wave of concern was amplified after AI researcher Jacob Coxon resigned from Anthropic, stating he was worried that leading AI companies are "gambling with our lives."
Anthropic's alignment lead further fueled the debate by sharing Coxon's post on X, adding, "We really do earnestly believe AI could kill all humans!" and personally estimating the chance at ">10% within the next decade." This statement came after increased capabilities were observed in recent models from Anthropic and OpenAI, and following a hack of an OpenAI internal model.
On TechCrunch's Equity podcast, hosts Kirsten Korosec, Sean O’Kane, and Anthony Ha discussed these warnings. Korosec speculated whether the doomer narratives might be a way for companies to "flex" their advanced AI models as they prepare for public offerings. O'Kane questioned how such concerns would be addressed in Anthropic's S-1 filing for an IPO.
Ha offered a counterpoint, arguing that while an existential threat might warrant an exclamation point, the use of "we" and the specific ">10%" figure were questionable. He also praised Coxon for putting his career on the line by resigning, a move distinct from executives who voice similar concerns without altering their professional trajectory.
In the wake of these discussions, Anthropic CEO Dario Amodei published a plan advocating for more cautious AI development.