Amanda Hoover
Students have submitted more than 22 million papers that may have used generative AI in the past year, new data released by plagiarism detection company Turnitin shows.
A year ago, Turnitin rolled out an AI writing detection tool that was trained on its trove of papers written by students as well as other AI-generated texts. Since then, more than 200 million papers have been reviewed by the detector, predominantly written by high school and college students. Turnitin found that 11 percent may contain AI-written language in 20 percent of its content, with 3 percent of the total papers reviewed getting flagged for having 80 percent or more AI writing. (Turnitin is owned by Advance, which also owns Condé Nast, publisher of WIRED.) Turnitin says its detector has a false positive rate of less than 1 percent when analyzing full documents.
ChatGPT’s launch was met with knee-jerk fears that the English class essay would die. The chatbot can synthesize information and distill it near-instantly—but that doesn’t mean it always gets it right. Generative AI has been known to hallucinate, creating its own facts and citing academic references that don’t actually exist. Generative AI chatbots have also been caught spitting out biased text on gender and race. Despite those flaws, students have used chatbots for research, organizing ideas, and as a ghostwriter. Traces of chatbots have even been found in peer-reviewed, published academic writing.
Undeniably, teachers are inclined to ensure that students do not misuse generative AI without disclosure or the necessary permissions. Providing evidence for such usage, however, can be challenging. Numerous educators resorted to sketchy, unverified enforcement methods, causing student unrest. Adding to this complexity, some educators even use generative AI in grading, further muddling the waters.
Determining the use of generative AI can be tricky as it doesn’t mirror the process of pinpointing plagiarism given that the generated text still presents as unique content. Additionally, the application of generative AI varies among students; while some might use AI to build their essays in bulk or entirely, others might use it as a tool for brainstorming or assistance.
Besides big language models like ChatGPT, students also find allure in alternative AI software known as word spinners that can recreate text, possibly concealing instances of plagiarism or AI-generated work. Annie Chechitelli, Chief Product Officer at Turnitin, states that their AI detector has been updated to detect word spinners. The detector can also flag work that has been rephrased by services like Grammarly’s spell checker, which now includes its own generative AI feature. The addition of generative AI components in regularly used software blurs the lines between what resources students can and cannot utilize.
However, bias is a potential risk within detection tools. Learners of English might trigger these detectors more frequently. A 2023 study found that seven different AI detectors produced a 61.3 percent false positive rate when evaluating Test of English as a Foreign Language (TOEFL) exams. Yet, this study didn’t include Turnitin’s model. Turnitin claims to have tailored its detector to examine written works from both native and non-native English speakers. An October study found Turnitin to be one of the most accurate among the 16 AI language detectors, as tested on undergraduate and AI-produced papers.
Reece Rogers
David Gilbert
Scott Gilbertson
Andy Greenberg
Schools using Turnitin were able to trial the AI detection software without charge, which concluded in early this year. Despite the risks of false positives and bias against English learners, a majority of Chechitelli’s clients have decided to acquire the AI detection. Some universities, however, due to these concerns, have decided to suspend utilising these tools. For instance, Montclair State University in New Jersey announced in November the discontinuation of using Turnitin’s AI detector, a decision also made by Vanderbilt University and Northwestern University during the last summer.
Emily Isaacs, Executive Director of the Office of Faculty Excellence at Montclair State, acknowledges the challenge. She indicates that the University is wary about biased outcomes from AI detectors, along with the fact that these tools fail to provide a confirmation akin to the one they get from plagiarism checks. Isaacs further states that Montclair State does not want an outright prohibition on AI, given its potential and inevitable role in academia. However, as trust builds in these tools over time, the policies might change. “This is not a forever decision, it’s a now decision,” she adds.
Chechitelli opines that the Turnitin tool should not be the only factor when grading a student. On the contrary, it presents an opportunity for faculty members to interact with students regarding the complex aspects of using generative AI. There is a general ignorance about the boundaries of its use, she mentions.
