The online network X has caused widespread outrage after users employed its artificial intelligence assistant Grok to manipulate pictures of women by taking off their attire.
Numerous instances have been seen where the technology has been employed to digitally undress women to render them in swimwear without consent, as well as to insert them in sexually explicit scenarios.
Developer Response and Legal Position
XAI, the organization responsible for Grok, did not respond to inquiries, aside from an auto-reply response stating "traditional news outlets lies".
A Home Office spokesperson stated that new laws is being proposed to prohibit nudification tools. Under a proposed law, anyone who supplied such technology would "receive a jail term and significant penalties".
"X or Grok has the ability to stop these forms of abuse if they wanted to," said a law professor, adding they "seem to have impunity."
Oversight Body Stance and Tool Functionality
The communications watchdog Ofcom said technology companies must "consider the danger" of people viewing illegal content on their networks, but did not confirm whether it was presently investigating X or Grok in regarding AI-generated images.
Grok is a free chatbot—with some premium functions—which answers X users' requests when they mention it in a post.
It is frequently used to provide reactions or more background to other posters' remarks, but people on X are also can edit an uploaded photo through its artificial intelligence picture modification tool.
History of Criticism and Policy
The tool has been faulted for permitting users to generate pictures and clips with explicit content and sexualised material, and it was earlier accused of creating sexually explicit content of celebrities.
"The platform has been enabling the creation and sharing of these pictures for months without taking any steps and we have yet to see any challenge by regulators," Professor McGlynn added.
XAI's own acceptable use policy prohibits "showing likenesses of people in a sexually explicit manner".
In a comment, Ofcom said it was unlawful to "make or distribute private pictures or child sexual abuse material" and confirmed this covered sexual deepfakes created with AI.
It said networks such as X were mandated to take "suitable measures" to "lessen the risk" of users viewing unlawful content on their platforms and to delete it swiftly when they become aware of it.
