Key facts
- A proposed class action lawsuit alleges xAI obstructed investigations into child sex abuse material (CSAM) generated by its Grok AI chatbot.
- One case involves a user who allegedly created 7,000 explicit images of his stepdaughter using Grok.
- xAI is accused of refusing to provide user information to law enforcement, hindering investigations.
- Stability AI was added as a defendant, accused of its models being used to create CSAM.
- Elon Musk has denied Grok has been used to generate child sex images.
A proposed class action lawsuit has been expanded to accuse X and its AI subsidiary xAI of obstructing investigations into child sex abuse material (CSAM) generated by the Grok chatbot. The amended complaint details a horrific case where a user allegedly created approximately 7,000 explicit images of his stepdaughter, who was 11 years old at the time, using Grok.
The lawsuit claims that Grok allowed the user to generate extreme content depicting incest and rape without adequate safeguards, only flagging a "gang rape" prompt which triggered a report to the National Center for Missing and Exploited Children (NCMEC). However, xAI is accused of repeatedly refusing to cooperate with law enforcement and NCMEC, withholding crucial user information such as IP addresses, thereby obstructing the investigation for weeks.
This alleged obstruction occurred even after the user was arrested, and a forensic review of his devices revealed the extensive CSAM. The complaint suggests that without Grok's "undressing" capabilities, the user might not have created such harmful images, which he allegedly trafficked online. The stepdaughter, identified as Jane Doe 4, has suffered severe emotional distress, anxiety, depression, and suicidal thoughts as a result of the exploitation.
Lawyers for the plaintiffs stated that Jane Doe 4's case is not isolated, citing NCMEC findings that 90 percent of xAI's CyberTipline reports were not actionable due to the company's refusal to include identifying user information. The suit further alleges that xAI prioritizes profits over child safety, with the feature used for generating explicit images now being a paid service.
Stability AI has also been added as a defendant, accused of training its open-weight models on CSAM and enabling third-party "nudify" apps that users allegedly employ to further alter Grok's outputs. The lawsuit claims that Stability AI removed safeguards against NSFW content to increase model usage. Researchers have identified the Stable Diffusion family as a primary driver of image-based nudification online.