The Alarming Failure of AI-Fakes Detection and Its Impact on Voters in the Global South

Recently, Donald Trump, a former president and convicted felon, shared several photographs that seemed to depict Taylor Swift’s supporters endorsing his campaign for the U.S. presidency. These images, which appeared to be AI-generated, were investigated by WIRED, who utilized True Media’s detection tool. This tool confirmed substantial evidence of manipulation in the photos.

The engagement with generative AI technologies, including their use in political arenas, is growing, as reported by WIRED in their coverage of AI’s role in global elections. However, outside of the US and some parts of Europe, the detection of AI-created content is complicated due to biases in the development of detection systems, making it arduous for journalists and researchers to contend with the surge of misinformation.

The field of media detection, particularly media manipulated or created using AI, is still developing in response to the rapid emergence of generative AI businesses. In 2023, these companies amassed over $21 billion in investments. “There are more available tools and technology that enable the creation of synthetic media compared to those that can detect it,” explained Sabhanaz Rashid Diya, founder of the Tech Global Institute.

Current media detection tools generally offer an 85 to 90 percent confidence level in identifying AI-generated content. However, according to Sam Gregory, program director at the nonprofit Witness, these tools’ effectiveness significantly decreases in non-Western contexts, such as in Bangladesh or Senegal, where the subjects might not be white or may not speak English. “The tools were developed with a focus on certain markets, prioritizing English language or faces common in the Western world during the training of the models,” Gregory noted.

This suggests that AI models are primarily developed with data from Western markets, thus lacking the capability to identify elements outside those norms. Often, this challenge arises from the reliance on readily available data on the internet, predominantly in English. Richard Ngamita, founder of Thraets, a nonprofit focusing on digital threats primarily in Africa, notes that much of the local data is in physical form, which hinders the ability of AI models to learn from it unless it is digitized.

Limited data to adequately train AI models makes them prone to incorrectly label real content as AI-manufactured, or vice versa. Diya points out that common detection tools often misidentify text from non-native English speakers as AI because they weren’t trained on diverse datasets, leading to numerous false positives.

Furthermore, it isn’t just about models failing to identify different accents, languages, or syntax; the tools initially were calibrated using high-quality media. In regions like Africa, prevalent use of budget Chinese smartphones complicates matters as these devices typically produce lower quality photos and videos, confusing the detection models, as explained by Ngamita.

Gregory comments on the sensitivity of some models to background noise or media compression, which could skew results under practical conditions. He criticizes the publicly available tools, which are most accessible to journalists and fact-checkers, for their pronounced inaccuracies due to the biased representation in their training data and challenges posed by lower quality material.

Generative AI is not just responsible for creating manipulated media. Other forms of manipulated media, such as “cheapfakes” – which involve alterations like misleading labels or alterations to audio and video – are prevalent especially in the Global South. These can often be incorrectly identified as AI-generated by inaccurate models or untrained researchers.

Diya expresses concern over the likelihood of non-Western content being mislabeled as AI-generated, potentially leading to unwarranted policy measures aimed at non-existent issues. “There’s a huge risk in terms of inflating those kinds of numbers,” she explains, noting that developing new detection tools isn’t as simple as it might seem.

Creating a detection model, like other AI technologies, requires substantial resources like energy and data centers, which are scarce in many parts of the world. Ngamita, from Ghana, points out the challenge: “If you talk about AI and local solutions here, it’s almost impossible without the compute side of things for us to even run any of our models that we are thinking about coming up with.” The alternatives for researchers like Ngamita are limited: either pay for costly tools like those from Reality Defender, use less accurate free tools, or seek access through academic connections.

Currently, Ngamita’s team collaborates with a European university to verify content. They have been gathering a dataset of potential deepfake examples from across Africa, which Ngamita says is proving valuable to academics and researchers aiming to diversify the datasets used by their models.

But sending data to someone else also has its drawbacks. “The lag time is quite significant,” says Diya. “It takes at least a few weeks by the time someone can confidently say that this is AI generated, and by that time, that content, the damage has already been done.”

Gregory says that Witness, which runs its own rapid response detection program, receives a “huge number” of cases. “It’s already challenging to handle those in the time frame that frontline journalists need, and at the volume they’re starting to encounter,” he says.

But Diya says that focusing so much on detection might divert funding and support away from organizations and institutions that make for a more resilient information ecosystem overall. Instead, she says, funding needs to go towards news outlets and civil society organizations that can engender a sense of public trust. “I don’t think that’s where the money is going,” she says. “I think it is going more into detection.”

Total
0
Shares
Leave a Reply

Your email address will not be published. Required fields are marked *

Previous Article

Unveiling the Magic: The Legend of Zelda's First LEGO Set - Great Deku Tree 2-In-1 Now Shipping!

Next Article

Exciting Lineup for Pokemon Go September 2024: Raids, Community Day, Spotlight Hours, and More!

Related Posts