Elon Musk’s artificial intelligence company xAI has faced mounting international condemnation following revelations that users exploited its Grok chatbot to create sexualised “deepfake” images of women and children.
The controversy has prompted government investigations across multiple countries and sparked the first major regulatory crackdown on mainstream AI-generated explicit content.
Free access fuelled abuse
Grok, Musk’s answer to ChatGPT and other AI chatbots, operates through an integrated account on the X social network. Until last week, users could request image generation and editing by tagging the bot in posts, receiving photorealistic results within minutes.
The system was quickly weaponized by users who sent photos of women to the bot with requests to “put her in a bikini” or “take her clothes off.” Unlike similar services previously confined to niche websites, Grok brought AI-powered “nudifying” technology into the mainstream through social media integration—completely free of charge.
The scandal deepened when investigators discovered users generating sexualised images of minors. Others created inappropriate content featuring victims of recent tragedies, including women killed in the New Year fire at the Swiss ski resort Crans-Montana and a woman shot by an immigration officer in Minneapolis.
An analysis by Paris-based AI Forensics of over 20,000 Grok-generated images found that more than half depicted “individuals in minimal attire” – predominantly women – with two percent appearing to show minors.
Countries take action
Indonesia became the first nation to completely block Grok access on Saturday, with Malaysia following suit on Sunday. India reported that X had removed 3,500 posts and 600 user accounts in response to government complaints, according to an anonymous government source who spoke to AFP.
Britain’s media regulator Ofcom announced Monday it was investigating whether X violated UK laws regarding sexual imagery, with potential fines reaching 10 percent of global revenue.
“If X cannot control Grok, we will—and we’ll do it fast,” British Prime Minister Keir Starmer warned members of Parliament from his Labour Party.
France escalated the response on Tuesday when children’s commissioner Sarah El Hairy referred Grok’s generated images to prosecutors, the Arcom media regulator, and European Union authorities. Digital affairs minister Anne Le Henanff dismissed the company’s restriction of image creation to paying users as “insufficient and hypocritical.”
The European Commission has ordered X to preserve all internal documents and data related to Grok through 2026 as part of its investigation. The EU has been probing X for potential digital content violations since 2023.
“We will not be outsourcing child protection and consent to Silicon Valley,” Commission President Ursula von der Leyen declared Monday. “If they don’t act, we will.”
Company response mixed
X’s safety team posted on 3 January that the platform “takes action against illegal content… including Child Sexual Abuse Material (CSAM) by removing it, permanently suspending accounts, and working with local governments and law enforcement.”
Musk warned that anyone using Grok to “make illegal content will suffer the same consequences as if they upload illegal content.” However, he simultaneously made light of the controversy by resharing to his 232 million X followers a post featuring a toaster wrapped in a bikini, adding laughing emojis.
By 9 January, Grok restricted image generation and editing services to paying subscribers only.
Facing mounting criticism from politicians worldwide, Musk pushed back, claiming on 10 January that critics of X and Grok “just want to suppress free speech.”
The scandal represents the first major test of how governments will regulate AI-generated explicit content as the technology becomes more accessible and sophisticated.
