Elon Musk's social media network X has provoked major anger after users utilized its artificial intelligence assistant Grok to change photos of females by stripping their clothing.
Several cases have surfaced where the system has been used to virtually strip women to make them appear in swimwear without permission, as well as to insert them in inappropriate contexts.
Company Reaction and Legal Stance
XAI, the organization behind Grok, did not respond to inquiries, except for an auto-reply response stating "legacy media lies".
A government official stated that legislation is being proposed to prohibit nudification technology. Under a proposed law, anyone who distributed such technology would "receive a prison sentence and substantial penalties".
"X or Grok could prevent these forms of misuse if they wanted to," said Clare McGlynn, adding they "seem to have impunity."
Regulator Position and Platform Capabilities
The communications watchdog Ofcom said tech firms must "evaluate the risk" of people seeing unlawful material on their platforms, but stopped short of confirming whether it was at this time looking into X or Grok in relation to artificial intelligence pictures.
Grok is a free chatbot—with some premium functions—which responds to X users' requests when they tag it in a message.
It is frequently used to offer reactions or additional context to other users' comments, but people on X are also capable of modify an posted picture through its AI image editing tool.
Background of Criticism and Rules
The system has been condemned for allowing users to generate photos and footage with explicit content and sexualised content, and it was in the past accused of generating graphic material of celebrities.
"X has been enabling the generation and sharing of these pictures for months without taking any measures and we have yet to see any challenge by regulators," the expert added.
XAI's own usage rules forbids "showing images of persons in a pornographic fashion".
In a statement, Ofcom said it was unlawful to "make or circulate deepfakes or child sexual abuse material" and clarified this covered explicit AI images made with AI.
It said services such as X were obligated to take "necessary steps" to "lessen the risk" of users encountering illegal material on their platforms and to remove it swiftly when they are notified of it.
