California orders xAI to halt Grok’s non-consensual sexual deepfakes
California Attorney General orders Elon Musk’s xAI to immediately stop Grok from generating illegal sexualized deepfake images of women and children.
Elon Musk’s artificial intelligence company xAI is facing intense legal pressure after California’s top law enforcement officer ordered it to immediately stop the creation of sexualized deepfake images through its Grok chatbot.
California Attorney General Rob Bonta on Friday issued a cease and desist letter to xAI, accusing the company of enabling the production and distribution of illegal, non-consensual sexual content, including material involving children. The move follows a surge in reports alleging that Grok has been used to generate explicit images of real people without their consent.
“Today, I sent xAI a cease and desist letter, demanding the company immediately stop the creation and distribution of deepfake, non-consensual, intimate images and child sexual abuse material. The creation of this material is illegal. I fully expect xAI to immediately comply. California has zero tolerance for child sexual abuse material,” Bonta said in a release.
He described the scale of the problem as alarming. “The avalanche of reports detailing this material — at times depicting women and children engaged in sexual activity — is shocking and, as my office has determined, potentially illegal,” he added.
The controversy has erupted amid growing global scrutiny of generative AI tools and their misuse. Over the past several weeks, xAI has been under pressure from regulators worldwide after Grok allegedly began producing sexualized deepfake images of users without consent. Initially, the company maintained that responsibility lay with users who generated unlawful content, warning that they could face legal consequences.
However, xAI has since shifted its stance, introducing safeguards to limit misuse. The company recently blocked Grok accounts on X from editing images of real people in revealing clothing. X, the social media platform owned by Musk, also emphasized its enforcement policies.
“We take action to remove high-priority violative content, including Child Sexual Abuse Material (CSAM) and non-consensual nudity, taking appropriate action against accounts that violate our X Rules. We also report accounts seeking child sexual exploitation material to law enforcement authorities as necessary,” X said in a post from its Safety account.
Despite these measures, Grok’s problems have already triggered regulatory action abroad. The chatbot app has been blocked in Malaysia and Indonesia, while Indian authorities have sought an explanation from xAI. In the United Kingdom, ministers have warned of consequences if stronger controls are not implemented.
The issue has also taken on a personal dimension. Ashley St. Clair, a political commentator and the mother of one of Musk’s children, has accused xAI of being used to create sexualized, non-consensual deepfakes of her.
This is not the first time Grok has drawn criticism. Earlier this year, the chatbot sparked outrage after producing anti-Semitic content, including praising Adolf Hitler and even claiming “Hitler” as its own surname. It also stirred controversy by naming Musk, Donald Trump, and JD Vance among the “most dangerous people for America.”
With California now taking formal legal action, xAI faces mounting pressure to prove that its technology can be controlled before public trust erodes further.
