The online network X has sparked widespread backlash after users used its AI chatbot Grok to change images of women by stripping their clothing.
Multiple examples have surfaced where the technology has been employed to digitally undress women to show them in swimwear without consent, as well as to insert them in inappropriate contexts.
Company Response and Regulatory Position
XAI, the firm that created Grok, remained silent to questions, other than an standard message stating "legacy media lies".
A government spokesperson stated that legislation is being brought forward to prohibit nudification technology. Under a proposed criminal offence, anyone who distributed such tools would "be subject to a prison sentence and heavy penalties".
"The platform has the ability to stop these types of abuse if they chose to," said a law professor, adding they "appear to enjoy impunity."
Oversight Body Stance and Platform Capabilities
The communications watchdog Ofcom said platforms must "consider the risk" of people seeing illegal content on their platforms, but did not specify whether it was presently looking into X or Grok in relation to artificial intelligence images.
Grok is a no-cost chatbot—with some additional functions—which responds to X users' prompts when they reference it in a post.
It is commonly used to give commentary or additional context to other users' remarks, but people on X are also able to alter an posted picture through its artificial intelligence picture modification tool.
History of Criticism and Rules
The system has been condemned for permitting users to generate photos and clips with explicit content and sexually suggestive material, and it was earlier alleged to have been involved in making sexually explicit content of public figures.
"X has been permitting the generation and distribution of these images for a long time without taking any measures and we have yet to see any intervention by regulators," Professor McGlynn added.
XAI's own usage rules bans "showing likenesses of persons in a sexually explicit manner".
In a comment, Ofcom said it was unlawful to "make or distribute deepfakes or child sexual abuse material" and clarified this included sexual deepfakes made with AI.
It said services such as X were required to take "suitable actions" to "lessen the risk" of users viewing unlawful content on their services and to take down it swiftly when they become aware of it.
