The online network X has sparked significant backlash after individuals utilized its artificial intelligence assistant Grok to alter photos of females by taking off their clothing.
Several instances have emerged where the technology has been employed to digitally undress women to show them in bikinis without consent, as well as to place them in sexually explicit contexts.
Company Response and Legal Position
XAI, the company that created Grok, offered no comment to inquiries, aside from an auto-reply response stating "traditional news outlets lies".
A government representative stated that legislation is being introduced to prohibit nudification tools. Under a new law, anyone who provided such technology would "receive a prison sentence and significant fines".
"X or Grok has the ability to stop these forms of abuse if they chose to," said a law professor, adding they "operate with impunity."
Regulator Stance and Platform Capabilities
The communications watchdog Ofcom said platforms must "consider the potential" of people accessing illegal content on their services, but stopped short of confirming whether it was at this time examining X or Grok in regarding AI-generated images.
Grok is a free chatbot—with some premium functions—which responds to X users' prompts when they tag it in a post.
It is commonly used to provide commentary or more background to other users' remarks, but people on X are also can modify an posted picture through its AI image editing tool.
History of Controversy and Rules
The system has been criticised for permitting users to create images and videos with explicit content and sexualised material, and it was previously alleged to have been involved in making sexually explicit material of celebrities.
"X has been enabling the production and spread of these images for months without taking any action and we have yet to see any action by oversight bodies," Professor McGlynn added.
XAI's own terms of service forbids "portraying likenesses of persons in a graphic manner".
In a statement, Ofcom said it was against the law to "create or distribute deepfakes or illegal content" and clarified this applied to sexual deepfakes made with AI.
It said networks such as X were mandated to take "suitable steps" to "lessen the danger" of users viewing illegal material on their platforms and to delete it promptly when they learn of it.
