San Francisco’s City Attorney, David Chiu, has issued cease-and-desist letters to Apple and Google, demanding the removal of 13 apps from their app stores that utilize AI technology to create non-consensual nude images, often targeting women and girls. These face-swap apps purportedly enable users to produce explicit deepfake content, which Chiu describes as illegal and harmful.
Chiu argues that the tech giants are profiting from these apps, which violate California laws against supporting services that generate deepfake pornography. He emphasizes the responsibility of these companies to ensure their platforms do not facilitate sexual abuse and to enhance their moderation systems to prevent such apps from appearing in the future.
Despite existing developer policies prohibiting pornography and abuse, both Apple and Google have faced scrutiny for allowing these apps to operate within their ecosystems. Recent studies and reports indicate that these companies have made substantial financial gains from apps employing nudification technology. The City Attorney’s Office points out that many of these apps, which have been widely downloaded, even carry ratings suitable for children.
Google has claimed to have removed hundreds of apps with nudifying features and has enacted measures to restrict their accessibility. Apple, on the other hand, has stated that responsibility lies with developers to adhere to guidelines, and that it actively engages in moderating its platform.
The proliferation of "nudification" applications has been alarming, with researchers noting that a significant number have the capability to create sexualized images from innocent photos. Such technology has raised concerns about the potential use of these images to bully and harass individuals, particularly women and girls.
The 13 apps under investigation disguise their true purpose as mere face-swapping tools, misleading users while actively promoting harmful content. San Francisco’s city attorney has made it clear that they will continue to address this issue and expects immediate action from Apple and Google. Should these companies fail to comply, Chiu’s office will explore all legal options available.
The city’s effort highlights the urgent need for tech companies to take accountability for the products that are distributed through their platforms, particularly as advances in AI technology continue to evolve.
