Elon Musk's online network X has caused major outrage after users utilized its AI chatbot Grok to manipulate photos of women by taking off their garments.
Numerous instances have been seen where the tool has been used to digitally undress women to show them in swimwear non-consensually, as well as to put them in inappropriate situations.
Developer Reaction and Legal Position
XAI, the firm behind Grok, did not respond to inquiries, other than an standard response stating "legacy media lies".
A Home Office spokesperson stated that legislation is being proposed to prohibit nudification software. Under a upcoming criminal offence, anyone who distributed such tools would "be subject to a jail term and significant penalties".
"X or Grok could prevent these types of abuse if they desired to," said a law professor, adding they "appear to enjoy a lack of consequences."
Oversight Body Position and Platform Functionality
The regulator Ofcom said platforms must "evaluate the risk" of people viewing unlawful material on their platforms, but did not confirm whether it was at this time looking into X or Grok in regarding AI-generated pictures.
Grok is a no-cost artificial intelligence—with some paid-for features—which reacts to X users' requests when they mention it in a post.
It is often used to offer reactions or further background to other users' remarks, but people on X are also can modify an posted image through its artificial intelligence picture modification function.
Background of Criticism and Rules
The system has been condemned for allowing users to create images and videos with explicit content and sexually suggestive content, and it was previously accused of generating inappropriate material of celebrities.
"The platform has been enabling the creation and sharing of these images for months without taking any measures and we have not seen any challenge by oversight bodies," the expert added.
XAI's own terms of service prohibits "depicting likenesses of persons in a pornographic way".
In a statement, Ofcom said it was unlawful to "generate or circulate non-consensual intimate images or child sexual abuse material" and confirmed this included sexual deepfakes created with AI.
It said networks such as X were mandated to take "necessary actions" to "lessen the risk" of users encountering illegal content on their services and to delete it quickly when they learn of it.
