San Francisco’s city attorney, David Chiu, has issued cease-and-desist letters to Apple and Google, demanding the removal of 13 "nudify" apps from their app stores. These face-swapping applications, which generate AI-created nonconsensual nude images, have raised significant concerns over their harmful impact on women and girls.
Chiu highlighted that generating such intimate images is illegal and unacceptable, calling attention to how these platforms have profited from the distribution of these apps. He noted the tech giants could have made millions from the fees associated with in-app purchases, while advocating for better moderation processes to prevent these apps from appearing in their stores.
The legal action cites state laws prohibiting support for services that create deepfake pornography. Despite existing policies from both companies against abusive and pornographic content, researchers have repeatedly identified numerous nudify apps within their platforms, some even rated for children. Previous investigations found up to 100 similar apps across both the App Store and Play Store, estimating combined revenues around $120 million.
Chiu’s office has taken a firm stance, emphasizing the psychological harm and bullying associated with the proliferation of sexualized deepfakes, which have affected numerous victims, some becoming suicidal as a result. He expressed that the tech companies must act quickly to eliminate these apps and strengthen their content moderation systems.
The 13 apps under investigation primarily function as face-swapping tools but also have features that allow users to create sexual deepfakes easily. The issue is compounded by a lack of protective measures within these applications, allowing for misuse.
Chiu stated, “This industry has a horrific impact on one’s reputation, mental health, and autonomy.” The city attorney remains committed to pursuing further action if necessary, urging both Apple and Google to fulfill their responsibility regarding the management of their platforms.
