The public method of the reviewer’s deterministic layer: eight bibliographic databases with a consensus rule, cross-checked DOIs, p values recomputed from their degrees of freedom, and what is not measured yet.
Every reference is checked against eight bibliographic databases and every p value is recomputed from its degrees of freedom. No model opinion involved: it is queried and calculated. This is what comes out when it is measured.
Reviewer3’s two public benchmarks, run with their code and their judge. Our numbers next to theirs and to the human reviewers’.
How it is measured, step by step
Measured on 2026-09-05 against the seven live databases: 120 articles sampled at random from Crossref (with their DOI and again without it), 40 books, 40 near-real ones with the authors swapped and 120 fabricated. The full list, with each verdict, is published.
Theses and local journals without DOI are not benchmarked yet.
312 results (t, F, χ², r, z) written in 13 different ways, measured on 2026-09-06: 144 with an altered p value and 168 intact.
75 hand-labeled sentences, measured on 2026-09-06: 35 with a planted disagreement and 40 correct but deceptive-looking (hyphenated ranges, intervals with their “95 %”, gas mixtures, 150 % increases). The cross-document layer, separately: 12 paragraphs, 100 % of planted disagreements caught and 100 % of correct ones with no false alarm. And real-life noise: 20 eLife preprints read in full, 2 flags within sentences (0 serious) and 1 across the document.
What it does not see: figures that only live in a table pasted as an image, and disagreements between the text and a results file (the full report does that with your output).
31 preprints with published peer review (eLife), measured on 2026-09-06: 385 objections from real reviewers, 266 of them major, against the 606 the full report wrote reading THE SAME manuscript they read, uncorrected.
How to read this: the tool reads the preprint exactly as the reviewers read it, without the authors’ response. eLife reviews what it agrees to review, so these are manuscripts that passed its triage, and in that model there is no accept/reject decision to compare against. Human reviewers are not the gold standard: this measures agreement, not truth. Extraction and matching are done by two independent models that did not write the report; there is no case-by-case human review.