Elon Musk's social media platform X has provoked major anger after users used its artificial intelligence assistant Grok to manipulate photos of women by taking off their garments.
Multiple examples have emerged where the tool has been employed to remove clothing from female subjects to render them in swimwear without consent, as well as to place them in inappropriate situations.
Company Response and Regulatory Stance
XAI, the company responsible for Grok, did not respond to questions, aside from an auto-reply message stating "traditional news outlets lies".
A Home Office official stated that legislation is being introduced to ban nudification software. Under a proposed criminal offence, anyone who distributed such tools would "receive a prison sentence and heavy penalties".
"X or Grok could prevent these types of misuse if they desired to," said Clare McGlynn, adding they "appear to enjoy a lack of consequences."
Regulator Position and Platform Functionality
The regulator Ofcom said technology companies must "consider the danger" of people seeing illegal content on their platforms, but did not specify whether it was at this time looking into X or Grok in regarding AI-generated images.
Grok is a free artificial intelligence—with some paid-for features—which reacts to X users' queries when they tag it in a post.
It is commonly used to offer commentary or additional background to other users' remarks, but people on X are also capable of modify an posted picture through its AI image editing tool.
Background of Controversy and Rules
The chatbot has been faulted for permitting users to produce photos and clips with explicit content and sexually suggestive material, and it was in the past alleged to have been involved in making inappropriate content of public figures.
"The platform has been enabling the production and spread of these pictures for a long time without taking any steps and we have not seen any challenge by regulators," Professor McGlynn added.
XAI's own terms of service bans "depicting likenesses of people in a pornographic fashion".
In a comment, Ofcom said it was illegal to "generate or circulate deepfakes or illegal content" and confirmed this included explicit AI images made with AI.
It said platforms such as X were mandated to take "necessary steps" to "reduce the danger" of users viewing unlawful material on their platforms and to delete it promptly when they learn of it.
