xAI can’t deny Grok makes CSAM anymore. So it’s suing users. - Ars Technica
Skip to content
AI
Biz & IT
Cars
Culture
Gaming
Health
Policy
Science
Security
Space
Tech
Forum
Subscribe
Story text
Size
Small<br>Standard<br>Large
Width
Standard<br>Wide
Links
Standard<br>Orange
* Subscribers only
Learn more
Pin to story
Theme
Search
Sign In
Sign in dialog...
Text<br>settings
Story text
Size
Small<br>Standard<br>Large
Width
Standard<br>Wide
Links
Standard<br>Orange
* Subscribers only
Learn more
Minimize to nav
Facing mounting pressure to acknowledge that Grok can still be used to generate non-consensual sexualized images of adults and minors, xAI filed a lawsuit Tuesday, suing the first user that Elon Musk’s firm has accused of using its chatbot to create illegal content.
The complaint targets Terry Wayne Harwood, who was arrested earlier this year for possession and distribution of child sexual abuse materials (CSAM), the South Carolina attorney’s office announced.
As xAI alleged, the company assisted in that arrest after discovering that Harwood had been using two xAI accounts for months to undress or “nudify” non-sexual images of multiple victims, including a young girl who appeared to be as young as 10.
xAI’s lawsuit comes a little more than a week after a different young girl joined a proposed class action representing several kids allegedly harmed by Grok. She alleged that her stepfather committed suicide after he was discovered using Grok, possibly in conjunction with other AI tools, to create 7,000 sexualized images of her and then distribute them on the dark web.
In that case, the victim alleged that xAI refused to help police identify the user who uploaded her image to Grok. To support claims that it is xAI’s common practice, her lawyers cited a 2026 National Center for Missing & Exploited Children (NCMEC) report confirming that 90 percent of xAI’s CyberTipline reports “were not actionable by law enforcement because xAI declined to include user information that would allow law enforcement to track and locate perpetrators.”
As victims accused X of shielding predators, Musk had previously maintained that he had not seen any examples of Grok-generated CSAM. Rather than move to restrict Grok’s outputs to make CSAM outputs impossible, Musk warned users to act responsibly, posting on X on January 3 that “anyone using Grok to make illegal content will suffer the same consequences as if they upload illegal content.”
While Musk was posting through it, though, Harwood was allegedly ignoring Musk’s warning and prompting Grok as many times as it took to get the chatbot to generate likely nonconsensual explicit images.
Now Musk seemingly can’t deny that Grok makes CSAM. xAI’s lawsuit claimed that Harwood used at least two accounts with convoluted usernames—“ceae2cb4-a9f6-4885-8ae9-6e2096d084f4” and “befccb94-4029-454d-9f1f-0d4945e8fa7c”—to generate illegal content from December 8 to February 18.
Sometimes Grok safeguards did prevent harmful outputs, “refusing to follow” some of the prompts “on the basis that such material violated Grok’s content moderation guardrails.” The lawsuit includes an example of an especially creepy prompt that Grok rejected, using phrases like “white slime” to mask intent to generate sexualized images.
That particular prompt may have been rejected for an obvious reason, though. Harwood explicitly asked the chatbot to “remove all her clothing,” which directly violates xAI user terms against requests to undress real people. In the proposed class action, it’s alleged that xAI overlooks a lot of bad requests, only reporting to NCMEC one prompt to depict “gang rape” out of 7,000 harmful outputs in the most recent victim’s case.
Likely to avoid other bad actors circumventing safeguards, xAI did not include examples of Harwood’s successful prompts or describe methods used to bypass filters. xAI only alleged that Harwood modified prompts to get around safeguards “in clear violation of the xAI Terms of Service and of US law,” including “some” requests for “obscene” images involving the “likeness of minor children.”
A spokesperson for the South Carolina attorney general’s office told Ars that Harwood’s case is still pending. Specifically, he has been charged with “distributing, transporting, exhibiting, receiving, selling, purchasing, exchanging, or soliciting CSAM that was ‘through the use of an artificial intelligence platform.’”
The spokesperson was not authorized to verify if Grok was the platform used. However, xAI’s complaint alleged that “upon information and belief, at least some of the images at-issue in the Harwood Criminal Action were generated or altered through Defendant’s violative use of Grok.”
xAI did not immediately respond to Ars’ request to comment.
xAI’s plan to blame users
In the lawsuit, xAI makes its case for why only users should be liable for Grok-generated CSAM. If the court agreed, such a finding could strengthen xAI’s defense in the looming...