Elon Musk's social media network X has caused significant backlash after users utilized its AI chatbot Grok to manipulate photos of females by removing their garments.
Numerous examples have been seen where the tool has been employed to remove clothing from women to render them in swimwear non-consensually, as well as to put them in inappropriate scenarios.
Company Response and Regulatory Position
XAI, the firm behind Grok, offered no comment to questions, aside from an automatically-generated message stating "legacy media lies".
A government spokesperson stated that legislation is being proposed to ban nudification technology. Under a proposed law, anyone who distributed such tools would "be subject to a prison sentence and substantial penalties".
"X or Grok is capable of halting these types of misuse if they wanted to," said Clare McGlynn, adding they "seem to have impunity."
Oversight Body Stance and Platform Capabilities
The regulator Ofcom said technology companies must "assess the danger" of people viewing unlawful material on their networks, but did not specify whether it was currently looking into X or Grok in relation to AI-generated pictures.
Grok is a no-cost artificial intelligence—with some paid-for functions—which responds to X users' queries when they mention it in a message.
It is often used to provide commentary or more background to other users' comments, but people on X are also capable of alter an uploaded image through its artificial intelligence picture modification feature.
Background of Controversy and Rules
The system has been faulted for allowing users to create images and clips with nudity and sexualised material, and it was earlier accused of making graphic content of celebrities.
"X has been enabling the production and sharing of these images for a long time without taking any steps and we have yet to see any intervention by regulators," the expert added.
XAI's own acceptable use policy prohibits "showing likenesses of people in a pornographic manner".
In a statement, Ofcom said it was unlawful to "generate or circulate deepfakes or child sexual abuse material" and confirmed this applied to explicit AI images created with AI.
It said services such as X were mandated to take "necessary measures" to "lessen the risk" of users viewing illegal content on their services and to delete it swiftly when they are notified of it.