Malaysia and Indonesia have taken action to block the artificial intelligence chatbot, Grok, developed by xAI, a company founded by Elon Musk. Concerns have arisen globally that the chatbot is being misused to create sexually explicit and nonconsensual images. This move comes amidst increasing scrutiny of generative AI tools capable of producing realistic visuals, audio, and text, raising doubts about the effectiveness of current safeguards in preventing abuse.
Grok, accessible through Musk’s social media platform X, has faced criticism for generating manipulated images, including inappropriate depictions of women and children. In response to global backlash over sexualized deepfakes, Grok recently restricted image generation and editing to paying users, but critics argue that this action did not adequately resolve the issue.
Regulators in Malaysia and Indonesia expressed concerns that the existing controls were ineffective in curbing the spread of fake pornographic content, particularly involving women and minors. Consequently, Indonesia temporarily blocked access to Grok on Saturday, followed by Malaysia on Sunday.
The Indonesian Communication and Digital Affairs Minister, Meutya Hafid, stated that non-consensual sexual deepfakes are a severe violation of human rights and citizen safety in the digital realm. The ministry emphasized the need to protect women, children, and the broader community from AI-generated fake pornographic content.
In Kuala Lumpur, the Malaysian Communications and Multimedia Commission imposed a temporary restriction on Grok due to repeated misuse of the chatbot to create obscene and non-consensual manipulated images, including content featuring women and minors. The regulator highlighted that demands for stronger safeguards from X Corp. and xAI were primarily met with reliance on user reporting mechanisms.
Launched in 2023, Grok is a free chatbot on X, enabling users to ask questions and create or respond to posts. Last summer, Grok introduced an image generator feature, Grok Imagine, with an adult content-generating “spicy mode.” The restrictions in Southeast Asia coincide with increasing scrutiny of Grok in other regions, including the EU, UK, India, and France.
In response to the concerns raised, the UK’s media regulator, Ofcom, has initiated a formal investigation into Grok’s compliance with its obligations to protect UK residents from illegal content. The regulator highlighted that Grok-generated images depicting children in sexualized contexts may constitute pornography or child sexual abuse material.
In contrast, Canada is not considering a ban on X, according to AI Minister Evan Solomon, who indicated ongoing discussions regarding the deepfake controversy.