The UK’s online safety regulator Ofcom has opened a formal investigation into X over concerns that its Grok AI chatbot has been used to generate and disseminate non-consensual intimate images and sexualized depictions of children — content that may constitute illegal intimate image abuse and child sexual abuse material (CSAM) under UK law. The investigation will examine whether X complied with duties under the Online Safety Act, including risk assessments, removal of illegal material, privacy protections, child-safety assessments and “highly effective” age assurance for pornography.
An Ofcom spokesperson described the reports as “deeply concerning” and said the inquiry would proceed “as a matter of the highest priority,” while stressing that Ofcom is not a censor and does not order removals, but judges whether platforms have taken “appropriate steps” to mitigate harm. X could face fines of up to £18 million or 10% of global turnover, as well as potential court-ordered business disruption measures in cases of serious non-compliance.
The move comes amid escalating global action against Grok. According to The New York Times, Indonesia and Malaysia temporarily blocked access to the chatbot over the weekend following public outrage over explicit deepfakes generated from real people. Indonesian officials called non-consensual sexual deepfakes a human rights violation, while Malaysia also cited child-safety concerns.
Elsewhere in Europe, Italy’s data protection authority issued a public warning about Grok and similar AI services, including ChatGPT, stating that such platforms enable users to “undress” real individuals without consent, posing “serious violations of the fundamental rights and freedoms of the individuals involved.” The regulator said it is coordinating with its Irish counterpart, which oversees X in the EU, and may take further action.
France has also moved into enforcement territory. Politico reported that the Paris prosecutor’s office opened an investigation into sexually explicit deepfakes generated by Grok after lawmakers filed complaints. Hundreds of women and teenagers reported being “undressed” through synthetic imagery posted to X. French officials characterized the content as “manifestly illegal” and referred cases to the Pharos cybercrime platform for removal.
In India, the controversy triggered executive intervention. India Today reported that X blocked roughly 3,500 posts, removed more than 600 accounts and assured the government it would comply with national law after the Ministry of Electronics and Information Technology warned that Grok had been used to generate obscene imagery targeting women and minors. Officials said X “accepted its mistake” and pledged to strengthen safeguards.
Ofcom confirmed it has also sought clarification from xAI, the developer of Grok, regarding potential compliance obligations under UK law. The regulator said it will provide further updates as the investigation progresses.
Need Help?
If you have questions or concerns about any global guidelines, regulations and laws, don’t hesitate to reach out to BABL AI. Their Audit Experts can offer valuable insight, and ensure you’re informed and compliant.
