Elon Musk has claimed a victory for his Grok app, noting it is the most downloaded in the UK, even as the government threatens to ban his platform X. The threat of a ban comes after it was discovered that Grok was being used to generate deepfake sexual images of women and children. Musk has dismissed the government’s concerns as an attempt to “suppress free speech,” setting the stage for a major regulatory showdown that could have significant implications for the future of social media regulation.
The Grok AI tool has been used to create highly disturbing content. Users have generated images of women and teenage girls in micro-bikinis, as well as in violent scenarios involving physical abuse and bondage. The potential for the tool to create child sexual abuse material has alarmed experts and led to calls for immediate action. The abuse faced by women targeted by these images has been described as severe and widespread, with many victims feeling powerless to stop the dissemination of the fake images.
UK Technology Secretary Liz Kendall has warned that the government is prepared to block X if it does not comply with safety regulations. She highlighted the powers available under the Online Safety Act and stated that Ofcom is expected to announce action within days. Kendall made it clear that the government will not tolerate the platform’s refusal to remove illegal material. Her stance reflects a growing determination among governments to reassert control over the digital sphere and to protect citizens from online harm.
International leaders have also condemned the platform. Australian Prime Minister Anthony Albanese called the use of AI for sexual exploitation “abhorrent” and a sign of social media’s failure to act responsibly. In the UK, the issue has become a political flashpoint, with former PM Liz Truss attacking the government’s stance. However, the focus remains on the urgent need to protect victims from digital abuse. The debate has highlighted the need for a coordinated international response to the challenges posed by generative AI.
X has restricted access to Grok’s image generation for free users but continues to offer the feature to paid subscribers. The incident has also drawn attention to the wider market for “nudification” apps, with MPs calling for a total ban on such technology. Advertisements for these apps have been found on major platforms like YouTube, prompting calls for stricter ad policies and expedited legislation. The call for a ban is driven by the recognition that current laws are insufficient to deal with the unique threats posed by AI-generated content.