Key facts
- Anthropic warns AI models may soon improve themselves without human involvement.
- The company advocates for an industry-wide pause on AI development.
- This advancement could make it harder for humans to maintain control over AI.
- Anthropic plans to convene policymakers, researchers, and civil society to discuss AI safety.
- Anthropic recently revised its Responsible Scaling Policy, dropping a commitment to pause training unilaterally.
Anthropic has published a blog post warning that its AI models are advancing rapidly and may soon be capable of improving themselves without significant human involvement, a process termed recursive self-improvement. The company is advocating for an industry-wide pause on AI development, citing the potential for such advancements to outpace human control and the need for adequate safeguards. The authors note that over 80% of code merged into Anthropic's codebase is now written by Claude, and engineers are shipping approximately eight times more code per quarter than before 2025. While this threshold has not yet been crossed, it could arrive sooner than anticipated, making it harder for humans to maintain control. Anthropic proposes that a pause should occur if multiple well-resourced labs in different countries agree to stop under the same conditions, likely referring to competitors like OpenAI, Google, xAI, and Meta. The timing of this proposal, shortly before a potential IPO and amidst a competitive race among AI labs, has led some observers to question its motivations, with critics previously accusing Anthropic of using safety rhetoric for competitive positioning. Notably, Anthropic recently revised its Responsible Scaling Policy, removing a commitment to unilaterally pause training while competitors continued. The company states this was a pragmatic response to the evolving landscape, aiming for transparency and matching rivals' safety efforts. Anthropic plans to convene policymakers, researchers, and civil society to discuss the implications of recursive self-improvement.