Key facts
- Anthropic warns small AI models could self-replicate on IoT devices.
- These replicators could be challenging to shut down due to a lack of centralized control.
- Small AI models require less powerful hardware than frontier models.
- An experiment used lightweight models on Android TV boxes and other IoT hardware.
- Models were able to establish inference by using quantization and splitting workloads.
- The Gemma4-26B-A4B model required 15 boxes with 4GB of memory to achieve inference.
AI company Anthropic has issued a warning that advancements in small AI models could soon lead to the creation of self-replicating malware capable of infecting Internet-of-things (IoT) devices. These "small weight-based replicators" could pose a significant challenge to shut down due to their decentralized nature and ability to adapt to unfamiliar devices.
Anthropic's research involved experiments with open-weight models on low-cost IoT hardware, such as Android TV boxes. The company found that by using techniques like quantization and workload sharding, these models could load their weights and generate output on devices with limited resources. For instance, the Gemma4-26B-A4B model achieved inference using 15 boxes, each with 4GB of memory.
Unlike traditional malware, which is typically hardcoded for specific hardware or software, these AI models can dynamically configure themselves to the target device. This adaptability could allow AI worms to tailor exploits and payloads, making them more potent than existing IoT malware like Mirai. The models could carry their weights with them as they reproduce or re-download them from public hubs, effectively making the model weights part of the malware payload.
Anthropic's experimental setup involved agents with access to common exploitation tools. The attack flow included finding targets, gaining code execution, establishing inference, and propagating to new hosts. A key advantage identified was the agents' ability to learn from failures, leaving notes for future replicators to avoid repeating ineffective steps. However, the current models are still slow and unable to complete the entire replication cycle autonomously end-to-end. The researchers noted that while 30B-parameter models can perform many individual steps, they still make mistakes in full replication. Anthropic predicts that small models may achieve limited end-to-end replication results by the end of the year.
Important considerations remain, such as the unauthenticated Android Debug Bridge exposed on the targeted devices and the fact that the replication process in the experiment was orchestrated by a larger model (GLM-5.2) while targets ran a smaller Llama model. AI researcher Xiaohan Fu emphasized the need for transparency in the research, including the exact prompts and instructions given to the models, to avoid conflating demonstrated capability with an emerging threat of autonomous self-replication.