A p value is not an independent piece of data: it is computed from the test statistic and the degrees of freedom. If you write t(28) = 2.10, the p value is fixed, and it is .045. If the next number says p = .02, one of the three is wrong. And a reviewer, or the software some journals run before a manuscript reaches anyone, can know it without your data.
It happens more than people think. A study that examined more than 250,000 p values from eight psychology journals found that roughly half of the papers using significance testing had at least one inconsistent with its test statistic and degrees of freedom, and about one in eight had an inconsistency large enough to change the statistical conclusion. It is rarely bad faith: a number copied from an old output, a change of sample that did not propagate, a mistyped decimal.
This guide explains how to check each test in under a minute, with which formulas, which errors are minor and which grave, why they occur, and how to make sure none reaches the manuscript.
A p value is the probability, under the null hypothesis, of obtaining a statistic as extreme as the one observed or more. That probability comes from a reference distribution (t, F, chi-square, normal) and depends on only two things: the value of the statistic and the degrees of freedom. You need to know nothing else about the study. That is why it is verifiable from the text itself.
When a paper writes “t(28) = 2.10, p = .02”, any reader can compute which p goes with a t of 2.10 on 28 degrees of freedom. The answer is .045. The difference between .02 and .045 does not change the conclusion (both are significant), but it shows a number was mistyped, copied from another analysis or belongs to a different sample. And a reviewer who sees that starts to doubt the rest of the figures.
These cases are computed with the exact distribution, not estimated. They show the difference between a minor and a grave error.
For a correlation, the check goes through converting it to t: t = r√(df) / √(1 − r²), with df = n − 2. If you read r(48) = .28, p = .03, the equivalent statistic is t = 2.02 and the p value is .049. The .03 does not correspond. This is one reason it is worth always reporting degrees of freedom.
A statistic reported to few decimals stands for a range. If it says t(28) = 2.1, the true value may lie between 2.05 and 2.15, and the corresponding p between .040 and .050. A serious check does not flag a p inside that margin, only those outside it. That is why checking programs distinguish minor from grave inconsistencies and do not fire on every one-decimal difference.
If you want to check your own figures before submitting, this is enough. In every case use the two-tailed value unless you declared a one-tailed test.
There is almost always a simple explanation, and knowing it helps prevent it.
Three habits remove nearly all these errors. First: copy results straight from the program output into the manuscript, or generate tables with code, and do not type them. Second: when you change the sample, rerun all the analyses and replace all figures at once, not by hand table by table. Third: before sending, go through the manuscript once checking each statistic against its p value, even if only with the formulas above.
That third pass is the heaviest, which is why it is worth automating. Previum recomputes each p value from the statistic and degrees of freedom in the text and flags only those that fall outside the rounding margin, along with figures that contradict each other across sections and references that cannot be found in bibliographic databases. It does not replace your judgement, but it does in a minute what takes an afternoon by hand. There are more errors of this kind in our guide to the statistical errors a reviewer spots first.
Paste your results section, or the whole manuscript, and every p value is recomputed from its test statistic and degrees of freedom: t, F, chi-square, r and z. It is the check this guide describes, the statcheck one, with a cautious rule: if the text does not allow a decision, it does not accuse. Every p that does not match comes with the quoted sentence, the value you reported, the recomputed one and whether the difference changes the conclusion.
It also applies the GRIM test to means of integer variables with their n and cross-checks the total N between sections. Everything runs in your browser: nothing is sent to any server and no account is needed. If no statistic is recognized, check that it carries its degrees of freedom in brackets with the p right after it.
It means the p value written is not the one that corresponds to that statistic and those degrees of freedom. It can be checked without the data, because the two fix the p value. It can be a minor error, which does not change the conclusion, or a grave one, when it flips from significant to not significant or the reverse.
Some do by hand, and some journals use programs that do it automatically before review. Even when the reviewer does not, a p value far from what its statistic implies is noticeable and raises doubts about the other figures.
Reporting the exact value is what APA 7 and most journals recommend, except for very small values (p < .001). The exact value is also what allows the calculation to be checked. If you write only p < .05 the error is still there, but nobody can see it.
Go back to the original program output and check which of the three numbers is wrong. Fix the error and see whether other derived figures (effect size, confidence interval) depended on it. If the manuscript is already published, tell the editor and ask for a correction.
Yes, through rounding: a statistic with two decimals stands for a range, and the computed p can fall within a margin. A properly done check allows for that margin. That is why a mismatch of a thousandth is usually not an error, and one of several points is.
For single-coefficient tests (t or z with their degrees of freedom), yes. With likelihood-ratio tests or approximate degrees of freedom (Satterthwaite, Kenward-Roger) the check is less direct and is better reproduced in the software.