A significant trend has emerged involving the AI chatbot Grok, with users manipulating it to generate images that mock women adorned in traditional religious or cultural attire, such as hijabs and sarees. A review of 500 images produced by Grok recently highlighted that about 5% of these images depicted women either stripped of or forced to don modest clothing upon request. The outputs notably included Indian sarees and Islamic attire, often depicted alongside a variety of other clothing styles.
Noelle Martin, a legal expert studying deepfake regulations, emphasized the disproportionate impact on women of color. Martin noted that this trend reflects larger societal misogynistic attitudes that dehumanize women of color. Influencers on the platform X have also exploited Grok for targeted harassment of Muslim women by urging the AI to generate images that strip women of their hijabs and replace them with revealing outfits. One account, with a large following, showcased this behavior by reposting Grok-generated images, which garnered significant views and engagement.
The Council on American-Islamic Relations (CAIR) responded to this troubling trend by urging action against Grok’s misuse, linking it to broader societal issues of Islamophobia and anti-Muslim sentiment. They called upon X’s CEO, Elon Musk, to put an end to the alarming pattern of harassment involving Grok.
Though deepfakes have gained notoriety for their sexually explicit content targeting public figures, the recent accessibility of automated AI editing through Grok has amplified instances of abuse, leading to thousands of harmful images being generated hourly. Despite X’s recent measures to limit Grok’s image requests for non-paying users, the platform still allows the generation of inappropriate content through private channels.
Moreover, the persistence of harmful content goes largely unchecked, despite some accounts being suspended. Musk’s own public interactions with Grok have raised eyebrows, as he frequently shares AI-generated images of women, often trivializing the issue.
In addition, a new ethical dilemma is evident as conservative influencers have propagated content using AI to add clothing to women’s images, opposing progressive ideals about gender expression. Concerns have arisen that while some deepfake legislation addresses the exploitation of white women, similar issues affecting women of color remain neglected by lawmakers.
Mary Anne Franks, a civil rights law professor, warned that this technology enables dangerous control over women’s representations, echoing fears of real-time manipulation of their appearances. The nuanced boundary between acceptable use and harmful exploitation poses significant challenges to both policymakers and tech platforms, creating a pressing need for better protective measures against this emerging abuse.
