Key facts
- Anthropic researchers are publicly warning that AI could cause human extinction within the next decade.
- Some Anthropic employees believe their colleagues share these concerns but are not speaking out.
- Elon Musk and other figures have labeled the warnings a 'psyop' or 'setup'.
- Anthropic stated it builds models with strong safeguards and is transparent about AI risks.
- Gary Marcus expressed concern about AI-driven catastrophes like bioweapons and disinformation wars, rather than extinction.
Several researchers and staff members at AI startup Anthropic have publicly voiced concerns about the existential risks posed by advanced artificial intelligence, echoing a former colleague's resignation. They fear the technology they are building could become so advanced and dangerous as to cause human extinction within the decade, and assert that many of their colleagues feel the same but have not spoken out.
These warnings follow a viral thread from Anthropic researcher Jacob Coxon, who resigned Wednesday, stating that neither Anthropic nor its competitor OpenAI are building AI models responsibly and are "gambling with our lives." Anna Wang, who works on Artificial General Intelligence Safety at Anthropic, said Thursday that many people at the company want to slow down development to address the risks, noting, "There is not yet a viable scientific plan to solve risks from recursively self-improving AI."
Drake Thomas, another Anthropic employee, agreed that "Things are moving way too fast, we don’t have anywhere near the degree of assurance we’ll want for ASI [artificial superintelligence]."
In response, Elon Musk and other figures on X, formerly Twitter, have dismissed the chorus of concerns as a "setup" and a "psyop." Musk suggested the groundwork for this "psy op" had been prepared for a long time, calling Coxon's post "the match that lit the fire." This theory was amplified by Parker Thayer, a researcher at Capital Research, who floated the idea that Coxon's post was the start of a "VERY sophisticated and well-funded PR operation to get support for Democrats to regulate AI into oblivion." Bill Ackman, CEO of Pershing Square, quoted Thayer's post, calling it "Interesting."
An Anthropic spokesperson defended the company's strategy, stating, "We have always been transparent that AI will bring both enormous benefits and unprecedented risks. To address these risks, we continue to build models with some of the strongest safeguards in the industry."
Other Anthropic employees had posted their agreement with Coxon shortly after his resignation. Samuel Marks, who works on safety research, wrote that "AI developers believe their technology could cause human extinction (or similarly bad outcomes)" and that "the more senior the employee, the more concerned they are." Evan Hubinger, described as a lead in the company’s alignment division, said Coxon was "correct" and that the industry was falling behind in addressing the potential for apocalyptic outcomes, adding, "We really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade."
Some experts doubt the technology will become intelligent enough to cause an apocalypse. Gary Marcus, a scientist and AI voice, argued that AI is already causing harm and called for a boycott. He expressed worry about "risk of catastrophe" from "AI-generated pathogens, from wars started or escalated by AI-generated disinformation, from hacks that destroy critical infrastructure, and so on."