Most people picture AI content detection as a tool that recognises ChatGPT’s handwriting. It doesn’t. Apart from watermarking, which only works when the AI vendor plants the mark itself, every detector is making a statistical guess: does this text look more like the AI samples it was tuned on, or more like the human ones? The percentage it shows is a confidence score set against a threshold someone chose. It is not a measurement of who wrote the page. Once you know how the three families of detectors produce that number, you can tell when a score means something and when to ignore it.
The Short Answer: Three Ways Detectors Score Text
Every AI detector on the market uses one or more of these approaches:
- Zero-shot statistical tests. A language model measures how predictable the text is, because machine output tends to be more predictable than human writing. Perplexity and burstiness, DetectGPT and Binoculars belong here.
- Trained classifiers. A model is trained on labelled human and AI text and learns whatever features separate them. Most commercial tools, including GPTZero and Turnitin, work this way today.
- Watermarking. The AI vendor nudges its own model’s word choices into a hidden statistical pattern while generating, then checks for that pattern later. Google’s SynthID Text is the best-documented example.
The first two infer authorship from style, so they can be wrong in both directions. Watermarking checks for a signal that was deliberately planted, which makes it the only approach that comes close to proof. Its catch is that it only covers text from vendors who plant the signal.
Perplexity and Burstiness: Where AI Detection Started
Perplexity measures how surprised a language model is by each next word. After “the cat sat on the,” the model expects “mat,” so perplexity stays low; “the cat sat on the tax return” pushes it up. LLMs generate by picking likely tokens, so their output tends to score low. Burstiness measures how much that predictability varies across a document. People mix flat sentences with odd ones; model output is often evenly smooth.
GPTZero made perplexity and burstiness famous in early 2023. Most explainers miss GPTZero’s own update note on that article: as of autumn 2023, the company says it no longer uses perplexity and burstiness for detection, having migrated to a deep-learning architecture. A guide that tells you “detectors measure perplexity and burstiness” is describing a 2023 design.
The weakness was built in. Predictable text is not the same as machine text. Technical documentation, legal boilerplate, product specs and essays by second-language writers are all predictable for reasons that have nothing to do with AI.
Smarter Zero-Shot Methods: DetectGPT and Binoculars
DetectGPT (Mitchell et al., ICML 2023) rewrites small parts of a passage with a second model such as T5 and checks how the original model’s probability changes. Text a model generated itself tends to sit at a local peak, so almost every small rewrite makes it less likely. The authors call this negative curvature. Human text shows no consistent peak.
Binoculars (Hans et al., 2024) uses two related models, an “observer” and a “performer,” and divides the text’s perplexity by a cross-perplexity term: how surprising one model finds the other’s predictions. That corrects for topics that are predictable for everyone. The authors report results at a 0.01% false-positive rate, far stricter than consumer tools. Like other perplexity-based methods, it tends to flag heavily memorised text, such as famous passages, as machine-made.
Both methods need a model’s token probabilities, which makes them strong in research and harder to package as a product that keeps pace with every new model.
Trained Classifiers: What Most Commercial Detectors Actually Are
A trained classifier learns from examples: vocabulary, sentence rhythm, hedging habits, transitions. It is fast and easy to sell, but it only knows the models it was trained on. Each new model release can shift what AI text looks like, so vendors have to keep retraining.
The cautionary tale is OpenAI’s own AI Text Classifier. At launch in January 2023, OpenAI reported that it correctly labelled 26% of AI-written text as “likely AI-written” and wrongly flagged human text 9% of the time, and warned it was unreliable under 1,000 characters. OpenAI withdrew it on 20 July 2023, citing its low rate of accuracy. The company that built the model could not build a dependable detector for it.
Read vendor claims just as carefully. Turnitin’s widely quoted document-level false-positive rate below 1% applies only to documents its detector judges to be more than 20% AI-written. Below that level, Turnitin marks the score with an asterisk because its own testing found more false positives. Every headline accuracy number rests on a test set and a threshold.
Watermarking: The Only Method That Can Prove Anything
Watermarking works the other way round: the signal is added during generation instead of inferred afterwards. In the scheme Kirchenbauer and colleagues presented at ICML 2023, the model picks a pseudo-random “green list” of tokens before each word and gently favours it. A reader sees nothing unusual, but a detector holding the key can run a statistical test on a short passage and get an interpretable p-value.
Google DeepMind’s SynthID Text, described in Nature in 2024, uses a method called tournament sampling. The paper reports a live experiment across roughly 20 million Gemini responses in which users rated watermarked and unwatermarked answers about the same. Google has open-sourced it, and it ships in Hugging Face Transformers from version 4.46.0.
Google’s documentation lists the limits: the watermark is less effective on factual responses, where word choice has less freedom; confidence can drop substantially after heavy rewriting or translation; and it isn’t designed to stop motivated bad actors. The bigger limit is coverage. The SynthID Detector portal Google announced in May 2025 only finds SynthID marks, so a negative result says nothing about text from other models. No watermark does not mean a human wrote it.
Why Human Writing Gets Flagged as AI
The best-known evidence comes from Stanford. Liang, Yuksekgonul, Mao, Wu and Zou, publishing in Patterns in 2023, ran seven popular detectors on 91 human-written TOEFL essays and on essays by US eighth-graders. The detectors were near-perfect on the native-speaker essays. On the TOEFL essays they averaged a 61.3% false-positive rate: all seven flagged 19.8% of them, and 97.8% were flagged by at least one. When ChatGPT enriched the vocabulary of the same essays, the average false-positive rate fell to 11.6%.
That is perplexity at work: limited vocabulary looks predictable, and predictable looks machine-made. The same happens to:
- Short passages, with too few tokens for a reliable estimate.
- Formulaic genres: product descriptions, compliance copy, how-to steps, FAQ answers.
- Writing polished by grammar tools, which nudge prose toward standard phrasing.
- Well-known passages that models have effectively memorised.
For the full error-rate picture, see Are AI Detectors Accurate?
Why AI Text Slips Through Anyway
False negatives are just as structural. At NeurIPS 2023, Krishna and colleagues showed that running AI text through a paraphrasing model called DIPPER cut DetectGPT’s detection rate from 70.3% to 4.6% at a 1% false-positive rate, with meaning preserved. The same attack also evaded watermarking, GPTZero and OpenAI’s classifier. Their proposed defence, retrieval against the provider’s own log of generations, only helps whoever holds the logs.
Weber-Wulff and co-authors tested 14 tools, including Turnitin and PlagiarismCheck, for the International Journal for Educational Integrity and concluded they are “neither accurate nor reliable,” lean toward labelling output human-written, and do worse on obfuscated content. The RAID benchmark (Dugan et al., ACL 2024), with more than 6 million generations across 11 models and 11 adversarial attacks, found 12 detectors easily fooled by attacks, changed sampling settings and unseen models.
The studies’ numbers differ, but the pattern is consistent: detectors do well on clean output from models they know and degrade once text is edited, paraphrased or comes from a new model.
How to Read an AI Content Detection Score
“87% AI” usually means the classifier is 87% confident the text resembles AI text, not that 87% of the words came from a model. Three habits keep you out of trouble:
- Ask for the false-positive rate at the threshold in use. A detection rate alone is meaningless; any tool catches every AI text by flagging everything.
- Do the base-rate maths. As an illustration, a 1% false-positive rate across 500 genuinely human articles a year wrongly flags about five writers, who then have to prove a negative.
- Expect tools to disagree. Different training data and thresholds give different verdicts, as the Stanford gap between 19.8% and 97.8% shows. Disagreement means the text sits where none of them can decide.
What Google Actually Checks
For SEO, Google’s 8 February 2023 Search Central post settled it: Google focuses on the quality of content, “rather than how content is produced.” The line it draws is automation, including AI, used primarily to manipulate rankings. That is the scaled content abuse policy, which applies to low-value pages “no matter how it’s created.” Google’s generative AI guidance adds the practical steps: fact-check and review all AI output before publishing, including alt text and structured data, and consider telling readers how content was made where that makes sense.
None of that involves passing an AI detector. A page that sails through every detector can still be thin, inaccurate content the helpful content system filters out. A page drafted with AI and improved with first-hand testing, original data and expert judgement is what E-E-A-T rewards. The better question is what makes a page worth citing, covered in non-commodity content.
A Practical AI Content Detection Policy for Teams
If you manage writers, freelancers or an agency roster, this workflow holds up:
- Decide what you are protecting. If it’s contract compliance (you paid for human writing), ask for drafts, version history and interview notes. Provenance is evidence; a detector score is not.
- Use detectors for triage only. A high score from two different tools is a reason to check the draft history, never a verdict, and never on short or formulaic text.
- Edit for substance, not for the detector. Skip “AI humanizer” tricks; they change wording, not value. Spend the time on fact-checking, catching hallucinations and adding experience nobody else has.
- Gate quality before publishing. SEO Rocket’s AI writer builds this in: every draft must pass validation gates (at least 1,000 words, title and meta length limits) and runs through a repair loop if it fails. Then do an editorial pass with an AI content quality checklist.
- Measure outcomes, not origin. SEO Rocket’s AI Visibility feature, built on Ahrefs Brand Radar data, shows how many ChatGPT, Google AI Overviews, Gemini and Perplexity responses cite your site, and the real questions behind those citations. That tells you more than any detector score. Background: which sources each AI engine cites.
It’s the same playbook proven across 1,000,000+ ranking pages: the pages that last have something worth reading, whoever drafted them.
AI Content Detection FAQ
Can Google detect AI-generated content?
Google hasn’t published any AI detection step in its ranking systems, and its documentation says it rewards helpful content however it was produced. What it acts on is low-value content produced at scale to manipulate rankings.
Can an AI detector prove a text was written by AI?
No. Statistical and classifier-based detectors give a probability, and published studies show high error rates on non-native writing and paraphrased text. Only a watermark check by the vendor whose model planted the mark comes close to proof, and only for that vendor’s output.
Should I run my content through a detector before publishing?
You can, but don’t make it a quality gate. A clean score says nothing about accuracy or usefulness. If you want to compare tools anyway, our review of AI detectors covers the main options.
