AI Detector Review
Every detector markets a percentage. Every study measures something narrower. This guide is the decoder: how accuracy studies are built, why the same tool can score 99% in one test and 26% in another, and what the key public research actually found.
Why accuracy percentages mislead
A detector's score on any test set depends entirely on what's in the set. Four design choices move results dramatically:
- Base rate. If a test set is 50% AI / 50% human, a tool can look strong. On realistic mixes (say 10% AI), the same tool's false positives dominate its error profile.
- Text provenance. "AI text" generated with a polite prompt and light editing behaves nothing like raw model output. Tools look better on raw output.
- Model vintage. Detectors trained on GPT-3.5-era text degrade against newer models. A 2024 study's star performer may be 2026's laggard.
- Whose thermometer. Vendor-run evaluations choose the set; independent studies choose their own. The gap between them is the story.
That's why this site labels every number vendor claim / public study / our test — the label is the context.
The studies worth knowing (labeled, linked)
Stanford-led bias finding — the equity landmark
Seven commercial detectors misclassified 61.3% of TOEFL essays by non-native English speakers as AI-generated; comparable native-speaker essays drew near-zero false positives. Simple prompt perturbations cut the average false-positive rate by 49.7%. (Liang et al., "GPT detectors are biased against non-native English writers," Patterns, 2023 — public study)
The claimed-vs-observed gap
A 2025 review reports commercial detectors claiming 63–100% accuracy — and concludes even the most accurate cannot reliably distinguish AI from human text. (Cheng et al., 2025 — public study)
UCLA's blunt overview
One widely used tool correctly identified only 26% of AI-written text while falsely flagging 9% of human writing. (UCLA Humanities Technology, 2025 — public study)
Balanced-set stress test
A 2026 evaluation on 192 balanced texts found false-positive rates of 43–83% across commercial detectors. (Paper Checker Hub, Feb 2026 — public study)
"Maybe impossible"
University of Maryland TRAILS researchers argue the false-positive problem cannot be fully mitigated and high-stakes use deserves caution. (UMD TRAILS, 2026 — public commentary)
Institutional mirrors
University law libraries (e.g., University of San Diego, 2026) maintain public guides on detector reliability — useful evidence that the caution isn't fringe.
Reading checklist for any new study (or marketing page)
- Who funded it? Vendor studies go in the vendor-claim bucket regardless of polish.
- What's the test set? Size, AI/human balance, text sources, model versions. Undisclosed set = unverifiable claim.
- Is the false-positive rate reported separately? Accuracy without FP rate hides the harm that matters in classrooms.
- When was it run? Pre-current-model results age fast; look for the date before the number.
- Does it name the tools? Unnamed "a popular tool" findings (like UCLA's) are still useful for the category, but can't power a tool-vs-tool decision.
How we use this research
Our scorecard keeps three evidence tiers separate and never averages them. When a vendor's 99% meets a study's 43–83% false-positive range, we show both, labeled. When we eventually run our own benchmark, its procedure gets published on the methodology page — test set composition, prompts, dates — so it's checkable.
FAQ
Are AI detectors accurate at all?
As a category: partially, context-dependently, and worse than marketed. Independent studies consistently find lower accuracy and higher false-positive rates than vendor claims (see above). For any specific tool, look for a labeled, dated, independent test — or treat the number as a claim.
Why do vendors claim 99%+ while studies find far less?
Different test sets. Vendor evaluations typically use clean, recent-model, unedited output at favorable base rates. Independent sets use realistic mixes and edited text. Both numbers can be "true" while only one describes your classroom.
Do detectors work better on newer or older AI models?
Generally worse on newer ones — detection lags generation. Studies age quickly; check dates before citing.
Which single study should I cite in a dispute?
The Stanford bias study (Liang et al. 2023) for equity contexts, and the Cheng et al. 2025 review for the general claimed-vs-observed gap. Both are peer-reviewed and widely referenced. If your institution has its own published guidance, cite that too — internal sources land hardest.