The social media network X has caused major outrage after individuals used its AI chatbot Grok to alter pictures of women by taking off their attire.
Multiple cases have surfaced where the system has been employed to virtually strip women to make them appear in bikinis without consent, as well as to put them in sexually explicit contexts.
Developer Response and Regulatory Stance
XAI, the firm that created Grok, did not respond to requests for comment, except for an automatically-generated response stating "legacy media lies".
A government official stated that legislation is being introduced to ban nudification software. Under a new criminal offence, anyone who provided such technology would "receive a jail term and heavy fines".
"X or Grok has the ability to stop these types of abuse if they chose to," said a law professor, adding they "appear to enjoy impunity."
Regulator Stance and Tool Functionality
The regulator Ofcom said platforms must "assess the potential" of people accessing illegal content on their networks, but stopped short of confirming whether it was at this time examining X or Grok in concerning AI-generated images.
Grok is a free AI assistant—with some additional features—which reacts to X users' requests when they mention it in a post.
It is frequently used to give commentary or additional context to other posters' remarks, but people on X are also able to modify an posted picture through its AI image editing feature.
Background of Criticism and Policy
The tool has been condemned for enabling users to generate photos and clips with nudity and sexualised material, and it was previously accused of generating sexually explicit material of celebrities.
"X has been allowing the creation and distribution of these images for months without taking any action and we have yet to see any intervention by regulators," the expert added.
XAI's own usage rules prohibits "portraying likenesses of people in a sexually explicit manner".
In a statement, Ofcom said it was against the law to "create or distribute non-consensual intimate images or child sexual abuse material" and stated this included explicit AI images created with AI.
It said services such as X were required to take "necessary measures" to "reduce the risk" of users encountering illegal material on their services and to delete it promptly when they are notified of it.
