Key facts
- A new attack called HalluSquatting exploits AI hallucination to create botnets.
- The attack targets AI coding assistants and agents like GitHub Copilot and Gemini CLI.
- Hackers predict and register resource identifiers that LLMs are likely to hallucinate.
- Malicious code is embedded in these registered resources to install reverse shells.
- This method allows for large-scale infection without individual targeting.
- Potential applications include ransomware, DDoS attacks, and cryptocurrency mining.
Hackers can leverage a new attack method called "HalluSquatting" to assemble massive botnets by exploiting the inherent tendency of large language models (LLMs) to hallucinate resource identifiers. This novel pull-based attack targets AI coding assistants and agents, which routinely access external repositories and registries for code and resources. By predicting and registering identifiers that LLMs are likely to hallucinate, attackers can embed malicious instructions, such as reverse shells, within these resources. When an LLM attempts to retrieve a hallucinated resource, it inadvertently downloads and executes the malicious code, leading to widespread device infection without direct targeting. This technique bypasses the scalability limitations of previous prompt injection attacks. The researchers noted that LLMs struggle to accurately identify resource locations, with hallucination rates reaching up to 85% for popular repositories and 100% for trending skills. This flaw is present across major LLMs, including Gemini and GPT models. The attack capitalizes on predictable hallucination patterns, such as self-referential slugs where a repository name is treated as the owner. Attackers can then register these predicted names and seed them with malicious software. The potential consequences include large-scale ransomware campaigns, distributed denial-of-service (DDoS) attacks, and cryptocurrency mining operations. The attack's name is derived from "typosquatting," a similar tactic used to lure users to malicious domains or packages by mimicking popular ones.