The online platform X has caused widespread backlash after individuals utilized its AI chatbot Grok to manipulate pictures of women by stripping their clothing.
Multiple instances have surfaced where the tool has been used to remove clothing from female subjects to render them in bikinis without consent, as well as to place them in sexualized situations.
Developer Response and Legal Stance
XAI, the organization that created Grok, did not respond to requests for comment, except for an automatically-generated response stating "traditional news outlets lies".
A government official stated that legislation is being brought forward to prohibit nudification software. Under a upcoming law, anyone who supplied such technology would "receive a prison sentence and significant penalties".
"X or Grok has the ability to stop these forms of misuse if they wanted to," said Clare McGlynn, adding they "appear to enjoy a lack of consequences."
Oversight Body Position and Tool Capabilities
The communications watchdog Ofcom said platforms must "consider the risk" of people seeing illegal content on their networks, but did not confirm whether it was at this time examining X or Grok in relation to artificial intelligence images.
Grok is a no-cost artificial intelligence—with some paid-for functions—which responds to X users' prompts when they mention it in a tweet.
It is often used to offer commentary or further context to other posters' remarks, but people on X are also can modify an posted image through its artificial intelligence picture modification feature.
Background of Controversy and Rules
The system has been faulted for enabling users to generate pictures and footage with nudity and sexually suggestive content, and it was previously accused of generating graphic material of public figures.
"The platform has been enabling the production and sharing of these images for months without taking any measures and we have yet to see any action by regulators," Professor McGlynn added.
XAI's own usage rules prohibits "depicting images of people in a sexually explicit way".
In a comment, Ofcom said it was against the law to "generate or distribute non-consensual intimate images or child sexual abuse material" and confirmed this covered sexual deepfakes made with AI.
It said networks such as X were obligated to take "appropriate steps" to "lessen the risk" of users viewing unlawful content on their services and to delete it promptly when they become aware of it.
