A detector labels a paragraph “likely AI.” That may look decisive, but a classification score is not a complete history of how the paragraph was written. Before using the result, ask what the detector was tested on, which errors it makes, and what other evidence is available.
At a glance
- Detection is a measurement task with possible errors.
- A score is not an independently verified account of authorship.
- The proportion of AI text in the tested population changes how results should be interpreted.
Separate the label from the evidence
NIST’s GenAI evaluation program examines both generated content and systems intended to distinguish it. This is a useful reminder that detection itself needs evaluation. A label should come with information about the test conditions, rather than being treated as proof simply because it is numerical.
Ask whether the evaluation involved the same language, length, subject, and editing process as the text you are examining. A detector tested on long essays may behave differently on a short product description. A strong result in one setting does not establish performance in another.
Why the base rate matters
Consider a hypothetical set of 1,000 documents containing 100 AI-generated texts and 900 human-written texts. A fictional detector flags 90 of the AI texts and also flags 45 human texts. Of 135 flagged documents, 45 are false positives: one third of all flags.
This example uses a 90% true-positive rate and a 5% false-positive rate. Neither number describes a real detector. It demonstrates why a low-looking false-positive percentage can still produce many mistaken accusations when most documents are human-written.
Use a review process that can correct mistakes
Preserve drafts, notes, source material, and revision history when authors choose to provide them. Review the actual quality and evidence in the work. A document may contain factual errors regardless of whether AI helped write it, and a detector does not verify its claims.
Avoid treating one flag as a reason for an automatic consequential decision. Ask for an explanation, allow correction, and record uncertainty. If a tool cannot explain its score or provide relevant validation, that limitation belongs in the conclusion.
A practical comparison
| Hypothetical flag result | Documents |
|---|---|
| AI texts correctly flagged | 90 |
| Human texts incorrectly flagged | 45 |
| All flagged texts | 135 |
Before using the result
- Check false-positive and false-negative rates.
- Match the evaluation population to the intended use.
- Assess sources and writing quality separately from authorship claims.
Continue exploring: our seven-step source-verification workflow.
Sources and methodology
Reviewed October 17, 2026. WorldLensia explanatory analysis. Worked examples and the accompanying teaching figure are illustrative, not measured findings or product tests. Consult the linked source for definitions and revisions.
To evaluate the errors and confidence claims behind a detector result, see AI Probability Calibration: Does an 80% Score Mean What You Think? and Precision, Recall and Accuracy: Build an AI Evaluation That Matches the Job.