The online platform X has provoked significant outrage after individuals utilized its AI chatbot Grok to manipulate images of females by removing their attire.
Numerous examples have emerged where the system has been employed to digitally undress women to make them appear in bikinis non-consensually, as well as to put them in sexually explicit scenarios.
Developer Response and Regulatory Stance
XAI, the organization that created Grok, offered no comment to inquiries, other than an auto-reply message stating "legacy media lies".
A government representative stated that new laws is being brought forward to outlaw nudification tools. Under a new criminal offence, anyone who distributed such tools would "be subject to a prison sentence and heavy fines".
"X or Grok could prevent these forms of abuse if they desired to," said a law professor, adding they "appear to enjoy a lack of consequences."
Regulator Stance and Tool Capabilities
The regulator Ofcom said tech firms must "assess the risk" of people viewing illegal content on their platforms, but did not specify whether it was at this time looking into X or Grok in relation to AI-generated pictures.
Grok is a free artificial intelligence—with some additional functions—which answers X users' prompts when they mention it in a post.
It is commonly used to offer commentary or more context to other posters' remarks, but people on X are also can edit an uploaded photo through its AI image editing feature.
Background of Controversy and Rules
The chatbot has been criticised for permitting users to generate images and footage with nudity and sexualised content, and it was earlier alleged to have been involved in generating sexually explicit content of celebrities.
"The platform has been enabling the creation and distribution of these images for a long time without taking any measures and we have not witnessed any challenge by oversight bodies," Professor McGlynn added.
XAI's own acceptable use policy forbids "showing likenesses of persons in a pornographic fashion".
In a statement, Ofcom said it was unlawful to "create or distribute private pictures or child sexual abuse material" and confirmed this included explicit AI images generated with AI.
It said networks such as X were obligated to take "necessary steps" to "minimize the danger" of users coming across unlawful content on their platforms and to delete it promptly when they are notified of it.
