Do AI text detectors work?
By the Remove AI Watermarking team · Updated
Short answer: Not reliably enough to prove anything. AI text detectors guess from writing style, so they miss AI text that has been edited and they flag human writing that happens to be plain or predictable. OpenAI shut down its own detector in 2023 because of low accuracy.
Key facts
- OpenAI's AI Classifier correctly flagged only 26% of AI-written text in OpenAI's own tests and wrongly flagged 9% of human text. It was withdrawn in July 2023.
- A 2023 Stanford study (Liang et al., published in Patterns) found seven detectors labelled 61% of essays by non-native English writers as AI-written, on average.
- The same detectors were near perfect on essays by US eighth graders, which points to writing style, not AI, as the cause.
- Detectors do worse on short texts. OpenAI said its classifier was very unreliable under 1,000 characters.
- Light editing or rewording often changes a detector's answer, in either direction.
What detectors actually measure
Most detectors score how predictable the words are and how evenly the sentences flow. Chatbot answers tend to be smooth and predictable, but so is careful writing, formal writing and writing in a second language. That overlap is why false positives happen.
Why one score isn't enough
A single percentage hides the reasons. Our AI Text Detector lists each signal instead: stock phrases, hidden characters from chat apps, leftover Markdown, heavy em dash use, even sentence lengths, repetition and list-heavy structure. You can see what was matched and decide whether it matters.
Using a detector fairly
- Treat any result as a reason to ask questions, never as proof.
- Compare with the person's earlier writing and drafts, and ask how they wrote it.
- Be extra careful with people writing in a second language.
- Don't rely on short samples, lists or technical text, where AI and human writing look alike.
What about watermarks in AI text?
Google has published SynthID Text, which can watermark text from its own models. OpenAI has said it researched text watermarking but did not ship it in ChatGPT. Text copied from a chatbot usually carries no watermark, though it may contain invisible formatting characters.
Try it free, right in your browser
See the signals in your text