Elon Musk's xAI told to immediately stop Grok’s sexualized deepfake images of women and children

xAI has been sent a cease and desist letter by the California Attorney General Rob Bonta on Friday, demanding that the company stop creating and distributing AI-generated non-consensual sexualized images with its Grok AI chatbot.

“Today, I sent xAI a cease and desist letter, demanding the company immediately stop the creation and distribution of deepfake, non-consensual, intimate images and child sexual abuse material. The creation of this material is illegal. I fully expect xAI to immediately comply. California has zero tolerance for child sexual abuse material,” Bonta said in a release.

“The avalanche of reports detailing this material — at times depicting women and children engaged in sexual activity — is shocking and, as my office has determined, potentially illegal,” he added.

Over the last few weeks, xAI has come under global regulatory pressure after the company’s Grok chatbot began generating non-consensual deepfake sexualized images of users without their consent. The company had earlier taken the stance that users should be held liable for generating unlawful content with its chatbot and even warned of legal consequences for such users. However, it recently went ahead and implemented measures to prevent Grok accounts on X from allowing image editing of real people in revealing clothes.

“We take action to remove high-priority violative content, including Child Sexual Abuse Material (CSAM) and non-consensual nudity, taking appropriate action against accounts that violate our X Rules. We also report accounts seeking child sexual exploitation material to law enforcement authorities as necessary,” X had said in a post from the company’s Safety account.

Grok’s issues have already led to the chatbot app getting blocked in Malaysia and Indonesia. Meanwhile, Indian regulators have also sought a response from the company, while UK ministers have openly said that there would be consequences for X if the company did not take measures to curb Grok’s sexualized image generation streak.

Even the mother of one of Musk’s children, political commentator Ashley St. Clair, has also accused xAI of being used to generate sexualized non-consensual deepfakes of her.

However, this is not the first time that Grok has landed itself in controversy. The chatbot had earlier this year gone off on an anti-Semitic hate-spewing frenzy where it even went on to praise Adolf Hitler. Later on, the chatbot also claimed that its surname was “Hitler”. Grok didn’t even leave Musk untouched when it named him, along with Donald Trump and JD Vance, in a list of the most dangerous people for America.