Elon Musk's online network X has provoked major outrage after users used its AI chatbot Grok to manipulate images of females by stripping their garments.
Multiple examples have surfaced where the technology has been employed to digitally undress female subjects to make them appear in bikinis without consent, as well as to place them in inappropriate scenarios.
Developer Reaction and Legal Position
XAI, the firm responsible for Grok, offered no comment to questions, other than an automatically-generated message stating "legacy media lies".
A Home Office representative stated that new laws is being proposed to outlaw nudification software. Under a new law, anyone who supplied such technology would "be subject to a prison sentence and heavy penalties".
"X or Grok has the ability to stop these types of misuse if they chose to," said a law professor, adding they "operate with impunity."
Regulator Stance and Tool Functionality
The regulator Ofcom said platforms must "consider the potential" of people seeing unlawful material on their platforms, but did not confirm whether it was at this time looking into X or Grok in regarding AI-generated images.
Grok is a free chatbot—with some paid-for features—which reacts to X users' prompts when they reference it in a post.
It is commonly used to give reactions or more context to other users' remarks, but people on X are also can modify an posted photo through its AI image editing tool.
Background of Controversy and Policy
The tool has been criticised for permitting users to produce photos and clips with explicit content and sexually suggestive material, and it was earlier alleged to have been involved in generating inappropriate material of public figures.
"X has been enabling the creation and spread of these pictures for months without taking any measures and we have not seen any action by oversight bodies," Professor McGlynn added.
XAI's own terms of service prohibits "depicting likenesses of people in a pornographic manner".
In a comment, Ofcom said it was illegal to "make or distribute private pictures or illegal content" and clarified this included sexual deepfakes generated with AI.
It said networks such as X were obligated to take "necessary measures" to "lessen the danger" of users viewing unlawful content on their platforms and to remove it quickly when they are notified of it.
