A 2022 experiment at Vanderbilt University involving 75,000 student assignments found that Turnitin's AI detection feature may have incorrectly flagged 1% of papers as artificially generated. This false positive rate raises concerns about the reliability of AI detection tools in academic settings.
The study, which has not been formally published but was reported by university officials, highlights the challenges universities face as they grapple with the increasing use of generative AI by students. Turnitin, a widely used plagiarism detection service, introduced its AI writing detection feature in April 2023.
Experts caution that false positives can lead to unfair accusations against students, particularly those who are non-native English speakers or use formulaic writing styles. The Vanderbilt experiment underscores the need for careful human review before penalizing students based solely on AI detection scores.
