This month, USA Today released a significant report demonstrating how U.S. Immigration and Customs Enforcement (ICE) delayed disclosing critical information about its detention policies. To compile and analyze data on these policies, the authors relied on the Internet Archive’s Wayback Machine, which preserves web pages for the public good. However, the irony noted by Wayback Machine director Mark Graham is that USA Today Co., the entity that owns USA Today and over 200 other media outlets, prevents the Wayback Machine from archiving its articles.
Other journalism organizations have similarly restricted access to the Wayback Machine. Currently, 23 major news sites are blocking the web crawler used by the Internet Archive, including The New York Times and Reddit. While The Guardian allows the Wayback Machine crawler, it restricts access to its content, complicating public access to archived articles.
USA Today representatives assert that their actions are part of measures to block all web scraping, not specifically targeting the Internet Archive. The Guardian has echoed these sentiments, raising concerns about potential misuse by artificial intelligence companies of their content.
In response, individual journalists and advocacy groups like the Electronic Frontier Foundation have rallied together in support of the Wayback Machine. They’ve gathered over 100 signatures from noteworthy journalists, expressing the tool’s vital role in preserving journalistic history, especially as many physical newspaper archives close and local libraries struggle to maintain digital records.
Prominent advocates for the tool include Laura Flynn of The Intercept and Micco Caporale of the Chicago Reader, both of whom highlight its importance in their research and fact-checking, as well as its utility in union organizing efforts by tracking historical job listings.
News organizations blocking the Wayback Machine cite concerns over AI and copyright issues. For instance, The New York Times claims that its content on the Internet Archive is used without permission by AI companies, posing competition against them. Reddit has stated similar fears regarding AI’s potential misuse of archived data.
The Internet Archive, which has preserved over a trillion web pages in its 30 years of operation, faces significant challenges due to these restrictions. Legal battles have already been a part of its history, but the current trend of limiting access threatens its core mission—preserving the documentation of digital history.
The risks of a significantly weakened Wayback Machine extend beyond just archival loss; such limitations could impact accountability journalism and hamper the legal system, as archived pages serve as crucial evidence in many lawsuits.
Graham remains hopeful that some publishers might eventually reverse their decisions blocking access. He emphasizes that the growing trend of restricting public access to the web undermines society’s ability to gain insights into current events and historical data.
