Key facts
- Anthropic secretly monitored Chinese users of its Claude Code AI model.
- The tracking code was discovered by a security researcher using 'prompt steganography'.
- Anthropic stated the code was an experiment to prevent unauthorized resellers and distillation attacks.
- The AI firm removed the tracker after the hidden code was exposed.
- Alibaba banned its employees from using Claude Code due to security concerns.
Anthropic, an AI firm, has removed a tracker from its Claude Code product after a security researcher exposed hidden code that was secretly monitoring Chinese users. The researcher, known as 'Thereallo,' discovered the 'prompt steganography' technique, which flagged users' timezone, proxy, and potential connections to Chinese AI labs accused of distillation attacks.
Anthropic engineer Thariq Shihipar confirmed the tracker was added in March as an 'experiment' to prevent account abuse from unauthorized resellers and protect against distillation. He stated that stronger mitigations had since been implemented and that the company had intended to remove the code earlier.
Privacy advocates criticized Anthropic's explanation, viewing the incident as evidence that the company is willing to cross lines to surveil users. The situation also highlights the ongoing efforts by US firms like Anthropic to prevent Chinese AI companies from copying their models. Chinese firms have reportedly matched US model capabilities within months, with a recent free AI model from Zhipu AI outperforming Anthropic's Claude Opus in finding computer vulnerabilities.
Anthropic, along with OpenAI, is urging the US to consider distillation attacks as intellectual property theft. Senator Tim Scott has supported the need for clear export control policies to prevent China from gaining a technological edge. Chinese researchers have found substantial evidence of distillation in most Chinese AI models, with Alibaba's Qwen AI model showing mimicry of Claude.
In response to the controversy, Alibaba has banned its employees from using Claude Code for work, citing 'back-door risks' and classifying it as high-risk software. The company's decision stems from concerns about potential legal and compliance risks if they violate Anthropic's terms, unlike individual users who can more easily evade location blockers.
The researcher 'Thereallo' noted that Anthropic could have transparently alerted users to the tracking instead of using hidden code, which they argued undermines trust. The researcher emphasized that coding agents already operate in a sensitive area, and such hidden surveillance practices make other privacy claims harder to believe.