A number appears in the report: a certain percentage likelihood of being written by AI. Decisions are made based on that number, sometimes resulting in accusations. What is rarely questioned are the two most fundamental issues: how that number was measured, and how often it is wrong.
Two machines often mistaken for one
Similarity checkers work by matching your sentences to an index of existing documents. If there is a match, the source can be identified. AI detectors do not have any index. They guess based on statistical patterns of word sequences, then output a probability. The first points to evidence, while the second calculates stylistic similarity. Both appear in the same report, and therefore are often considered equivalent.
The creator itself withdrew the tool
OpenAI launched the AI Text Classifier on January 31, 2023, and closed it on July 20, 2023, citing low accuracy levels. The figures they announced themselves: the tool correctly identified 26 percent of AI-generated text, and incorrectly labeled human writing as AI in 9 percent of cases. Their official note added three rarely quoted limitations, namely that the tool should not be the primary decision-maker, is unreliable for texts under 1,000 characters, and performs much worse outside of English.
The most affected are non-native writers
Liang and colleagues in the journal Patterns (2023) tested seven popular detectors on 91 TOEFL essays by non-native writers and 88 essays from 8th-grade students in the United States. The results were skewed:
- average false positive rate on American student essays: 5.19 percent
- average false positive rate on TOEFL essays: 61.22 percent
- 18 out of 91 TOEFL essays (19.78 percent) were labeled as AI by all seven detectors
- 89 out of 91 TOEFL essays (97.80 percent) were labeled as AI by at least one detector
The cause is not cheating, but rather a more limited choice of words, making the text easier for machines to predict. The evidence is twofold. When the vocabulary of TOEFL essays was enriched, the false positive rate dropped from 61.22 to 11.77 percent. Conversely, when American student essays were simplified, the false positive rate increased from 5.19 to 56.65 percent. What the detectors measure is closer to linguistic richness than to the honesty of the writer.
The same research showed the opposite side. A single round of editing reduced the detection of AI-generated essays from 100 percent to 13 percent, and AI-generated scientific abstracts from 68 percent to 28 percent. Thus, those who intentionally try to evade detection are the easiest to pass.
Read the terms listed by the vendor
Turnitin openly lists its figures, and the terms also determine the meaning of those figures. A false positive rate below 1 percent only applies to documents with over 20 percent AI content. Scores of 1 to 19 percent are deliberately not displayed and replaced with an asterisk, precisely because that range is prone to false positives. At the sentence level, the false positive rate is around 4 percent, and 54 percent of those are attached to sentences that are indeed AI-generated. Short documents are predicted to be almost all or none at all. Only prose sentences are counted, so lists, short points, and code do not count.
One thing that is often overlooked in Indonesia. According to its official FAQ updated on August 28, 2026, Turnitin's AI detection only works for long manuscripts in English, Spanish, Japanese, and Modern Arabic. Manuscripts in Indonesian are not processed at all: the indicator is empty and the report does not publish. Other technical requirements for English manuscripts are a minimum of 300 words of prose and a maximum of 30,000 words.
What remains to hold on to
The vendor itself states that the tool does not determine the existence of violations, but rather presents data for human assessment. Thus, what matters remains the process: the version history of the document, initial drafts, research notes, and the writer's ability to explain the content of their writing. The detector's score is useful for opening a conversation, not closing it.
If you prefer to see the explanation directly, there is a short clip at What is AI Detector and Why Its Results Cannot Be 100% Trusted.
Sources
- New AI classifier for indicating AI-written text, OpenAI, January 31, 2023 with closure note July 20, 2023
- Liang W., Yuksekgonul M., Mao Y., Wu E., Zou J., GPT detectors are biased against non-native English writers, Patterns, 2023
- Turnitin's AI writing detection capabilities FAQs, Turnitin Guides, updated August 28, 2026
- Understanding the false positive rate for sentences of our AI writing detection capability, Turnitin, June 14, 2023