Direct answer
A 2023 study from Stanford HAI tested 7 leading AI detectors against 91 essays written by non-native English speakers. The detectors flagged more than half of the human-written essays as AI-generated, versus near-zero false positives on essays by native speakers. The gap persisted across detectors and was not explained by essay topic or length. The practical implication: a detector score on text written by a non-native English speaker should not be used as proof of AI authorship.
The Stanford HAI paper by Weixin Liang and colleagues, published in July 2023 in the journal Patterns, is the most-cited piece of research on fairness in AI content detection. It is quoted in nearly every subsequent write-up on detector reliability, including the AI detection explained guide on this site. What that study actually found, how the research was done, and what the figures mean for practical decisions by teachers, employers and students is the subject of this piece.
The headline numbers
| Writer group | Essays sampled | Flagged as AI (one or more detectors) | Flagged by all 7 detectors |
|---|---|---|---|
| Non-native English (TOEFL) | 91 essays | 61.3 percent | 19 percent |
| Native English (US 8th-grade) | 88 essays | 5.1 percent | 0 percent |
Figures reported in Liang et al. (2023), covering 7 leading AI detectors against two groups of human-authored essays. The study is available in full on Stanford HAI's website.
What the researchers actually tested
The study used two datasets of human-written essays. The first was a sample of 91 TOEFL essays by non-native English writers; TOEFL is the standard English-proficiency exam for university admissions in the United States. The second was 88 essays by US eighth-grade students, as a native-English baseline. The researchers ran both datasets through 7 publicly available AI content detectors: GPTZero, Originality.ai, Quil, Sapling, Crossplag, ContentAtScale and one unnamed detector. The sample is small by machine-learning standards and large enough to be statistically meaningful for the headline comparison.
The researchers did not use any AI-generated essays in the main experiment: all essays were human-written. Any flag the detectors returned was by definition a false positive. The published finding is that the false positive rate for non-native English writing was approximately 12 times the rate for native-speaker writing when any detector was allowed to flag, and that nearly a fifth of non-native essays were flagged by all 7 detectors agreeing.
Why the bias occurs
AI content detectors look for low perplexity and low burstiness - roughly, text that is too predictable and too uniform in sentence length. That signature happens to overlap with the signature of non-native English writing by second-language speakers who have a smaller active vocabulary and tend to use more common grammatical constructions. The detectors are not detecting the author; they are detecting a pattern that the author happens to share with the models the detectors were trained to flag. The authors of the study were explicit: this is not an incidental weakness of specific tools, it is a systemic property of the perplexity-and-burstiness class of detector.
The practical consequence is that treating a detector score as a verdict - rather than as a flag worth investigating - produces a predictable pattern of over-accusation against international students, English-as-a-second-language employees and anyone whose written English is grammatically correct but stylistically plain. The Stanford paper framed this as an equity issue in US universities; the same mechanism applies to any workplace or publication that treats detector scores as proof.
What to do about it in practice
If you teach or run an integrity office: do not treat a detector score as proof. The Stanford figures put the floor on false positives for a known subpopulation at roughly 60 percent; your detector will produce the same floor against your international students. Treat high scores as a flag that triggers a conversation, not a conviction. The AI detection explained guide covers the appropriate role of detectors in integrity policy.
If you are a non-native English writer being flagged: ask whoever is doing the flagging which detector was used, what the stated accuracy is on your specific population, and what the appeal process is. Cite the Stanford paper by name; it is the most-cited single piece of evidence on this question and most reasonable integrity officers will know it on sight.
If you are an employer or publisher making a decision based on detector output: set a policy up-front that a detector flag is a flag, not a finding. Follow it up with a written review by someone who can read the piece and judge it on its substance. The cost of a wrong accusation is high enough that an extra review pass is cheap by comparison.
Frequently asked questions
Has the research held up since 2023?
The Stanford paper has been cited in dozens of follow-up studies and is treated as the reference result on detector bias against non-native English writers. Detector vendors have since published updates claiming reduced false positive rates, but no independent replication at the same scale has shown the gap closed to zero. Treat the Stanford figures as a reasonable floor, not a historical artefact.
Which detectors were least biased?
The study reported the gap as broadly consistent across the 7 detectors it tested rather than as a single detector being much worse than the others. In other words, the problem is systemic to the perplexity-and-burstiness approach rather than a specific vendor failing. For a per-vendor read, the Turnitin vs GPTZero and Originality.ai review pieces are the best starting points.
Does running non-native English text through a humaniser help?
A cadence-aware humaniser can reduce false positives by varying sentence length and vocabulary against the signature detectors look for. It is not a guaranteed fix, and more importantly it does not address the underlying integrity question. The right first move is to challenge a detector-based accusation rather than to reshape your writing to defeat the classifier.