Elon Musk's social media network X has caused major anger after people utilized its AI chatbot Grok to alter photos of women by taking off their garments.
Multiple cases have surfaced where the technology has been employed to remove clothing from female subjects to render them in bikinis non-consensually, as well as to insert them in inappropriate situations.
Developer Response and Regulatory Stance
XAI, the firm behind Grok, offered no comment to questions, other than an automatically-generated message stating "traditional news outlets lies".
A government official stated that legislation is being proposed to prohibit nudification technology. Under a new criminal offence, anyone who supplied such tools would "face a prison sentence and significant penalties".
"The platform has the ability to stop these forms of abuse if they wanted to," said Clare McGlynn, adding they "seem to have a lack of consequences."
Oversight Body Stance and Tool Functionality
The regulator Ofcom said tech firms must "consider the danger" of people viewing illegal content on their networks, but did not specify whether it was currently looking into X or Grok in regarding AI-generated images.
Grok is a no-cost chatbot—with some paid-for features—which responds to X users' requests when they reference it in a tweet.
It is often used to provide reactions or more context to other posters' remarks, but people on X are also capable of edit an posted photo through its AI image editing tool.
Background of Controversy and Policy
The system has been criticised for allowing users to create pictures and clips with nudity and sexually suggestive material, and it was earlier accused of generating sexually explicit content of celebrities.
"X has been allowing the generation and sharing of these images for a long time without taking any action and we have not seen any intervention by regulators," Professor McGlynn added.
XAI's own acceptable use policy prohibits "depicting images of persons in a pornographic way".
In a comment, Ofcom said it was unlawful to "make or circulate private pictures or illegal content" and clarified this included sexual deepfakes made with AI.
It said platforms such as X were required to take "suitable measures" to "lessen the risk" of users viewing unlawful content on their services and to delete it quickly when they learn of it.
