Key facts
- Anthropic proposes a global system for collective decisions on pausing AI development.
- The system would involve governments and AI developers.
- The goal is to mitigate potential risks posed by advanced AI, including autonomous cyberattacks.
- Anthropic's AI models show increasing self-improvement capabilities.
- Current cybersecurity frameworks may not account for AI-enabled attack techniques.
AI company Anthropic has called for the establishment of a global system that would enable governments and artificial intelligence developers to collectively decide on when to slow down or pause work on AI technology. This proposal aims to address the potential risks associated with the rapid advancement of AI, particularly the increasing self-improvement capabilities observed in models like Anthropic's own Claude, and the growing danger of AI-enabled cyberattacks. The company suggests that such a coordinated approach is necessary to ensure that AI development does not outpace human control and that adequate safety measures are in place. Anthropic's analysis indicates that current cybersecurity frameworks, including the widely used MITRE ATT&CK framework, do not fully account for AI-enabled attack techniques. The call comes amid a competitive race among major AI labs to develop increasingly sophisticated models, raising concerns about the pace and direction of innovation.