A reviewer does not need to rerun your analyses to know something is off. Many statistical errors show on a first read, because they leave marks in predictable places: the abstract, Table 1, the paragraph that justifies the sample, the last sentence of the discussion. Someone who has refereed for years knows where to look.
Most of these errors are not serious in themselves and do not imply bad faith. They are inherited habits, outdated reporting templates, a sentence from the dissertation that slipped into the paper. But a reviewer who sees two or three in a row stops reading generously, and the tone of the report changes. Fixing them before you submit is the best-paying use of preparation time.
This guide goes through the twelve most frequent: what the reviewer sees, why it is a problem and how to fix it, with a link to the detailed guide for each. Use it as a checklist before you press submit.
These are the most visible in the discussion and the ones a reviewer corrects with the most impatience, because they depend on no software: they depend on how a result is read.
A p value above the threshold does not show the effect is zero: it says the data do not allow you to rule it out at that level of evidence, often because the sample is small. “There were no differences between the groups” is a positive claim that the result does not support. What you can write is that no detectable difference was observed, and add the confidence interval so the reader sees which values it still allows.
It is common to read “the treatment worked in group A (p = .03) but not in group B (p = .12)” and conclude that the groups differ. That one effect is significant and the other is not does not show they differ from each other; for that you have to test the difference directly, with an interaction or the interval of the difference. It is a classic error, described long ago in the methodological literature, and reviewers recognise it at once.
A cross-sectional study can describe associations; it cannot say one variable “causes”, “reduces” or “predicts” another in a temporal sense. A title with “effect of” on cross-sectional data invites rejection. Without manipulation or follow-up, the verb should reflect an association and the discussion should name the alternative explanations. The full guide is on the page about causal claims from cross-sectional designs.
A p value says whether there is something to detect; it does not say how much. Without an effect size the reader cannot judge whether the result matters, and without an interval they cannot tell how imprecise it is. Almost every psychology and health journal asks for them. The guide on missing effect sizes explains which measure to choose for each test.
A p value that does not correspond to the test statistic and degrees of freedom it sits next to, degrees of freedom that do not fit the sample size, a percentage that does not add up, a mean that is impossible for the number of participants. These are transcription or version errors, not method errors, and a reviewer finds them by recomputing. Each type is explained in the guides on p values that do not match the test statistic and on the GRIM test.
Dropouts, the share of incomplete cases or the range of the descriptive variables are missing. When they are, the reviewer cannot judge whether the sample is adequate or whether the analysis had to discard cases. A complete descriptive table and a participant flow diagram settle most of those comments before they exist.
There is no universal minimum size; there is a size sufficient for the effect you want to detect with the analysis you use. A sample presented without justification, or on the grounds that it is “similar to previous studies”, invites the comment that the study lacks power. Settle it with an a priori calculation or, if that is impossible, with a sensitivity analysis: which minimum effect does your sample detect. There is a guide on underpowered studies.
The software drops cases with missing values by default, and if you do not say so the reader assumes there were none. How many are missing, in which variables and with what pattern is information requested in any review. Multiple imputation is the usual alternative when deletion is not defensible, and the blog guide on it walks through the steps.
Students within classrooms, patients within centres, repeated measures from the same person. If you treat those observations as independent, standard errors come out too small and results look more certain than they are. A reviewer spots it as soon as they read how the sample was recruited. The fix is a model that accounts for the clustering, or a stated reason why it is not needed.
Twenty tests at a .05 threshold produce, by chance alone, some significant result. If the paper runs many comparisons without adjusting, the reviewer will say so, and be right. Pre-specify a primary outcome, correct (Bonferroni, Holm or false-discovery-rate control) or state openly that the analysis is exploratory.
Normality of residuals, homogeneity of variance, linearity, absence of multicollinearity. An ANOVA or a regression with no line on its assumptions is an invitation to ask for them. You do not need a chapter: one sentence with the check and, if it fails, which robust alternative you used. The blog guide on writing results when assumptions are violated shows how.
An unexpected result presented as though it had been the hypothesis from the start, an analysis chosen because it “worked”, a variable dichotomised at the most favourable cut-off. It is called HARKing and is often not deliberate. A reviewer perceives it from the excessive coherence between introduction and results. The honest way out is to separate the confirmatory from the exploratory.
You do not need to reread the manuscript twelve times. Do it in three passes. In the first, read only the abstract and the discussion looking for causal language and conclusions about null effects (errors 1 to 3). In the second, read results and tables with a calculator beside you: does each p correspond to its statistic, do the percentages add up, are there effect sizes and intervals (errors 4 to 6). In the third, read the method with a reviewer’s head: sample, dropouts, nested data, assumptions, corrections and what was pre-specified (errors 7 to 12).
The numerical checks, which are the most tiring to do by hand, can be automated: Previum recomputes p values from their degrees of freedom, flags figures that contradict each other across sections, and verifies each reference. It does not replace your judgement about the design, but it frees you from the work that generates the most human error.
Three of the errors on this list can be checked without a human reading the paper, with arithmetic alone: the p value that does not follow from its test statistic, the mean that is impossible for its n, and the N that changes from one section to another. Paste your results section, or the whole manuscript, and the check recomputes every p from t, F, chi-square, r or z, applies the GRIM test to means of integer variables and cross-checks the N between sections.
Every inconsistency comes with the quoted sentence, what does not match and the recomputed value. Everything runs in your browser: nothing is sent and no account is needed. The other errors on the list, the ones about design and interpretation, are what the simulated reviewers in Previum read in the whole manuscript.
Among the most repeated are missing effect sizes and confidence intervals, and conclusions that go beyond the design. They are easy to fix, which is why they are worth checking before you submit.
It depends which one. Reporting errors usually turn into a request for correction. Design or analysis errors that affect the conclusion (an insufficient sample, an analysis that does not answer the question, causality without a basis) can lead to rejection or major revisions.
Reporting and numerical-coherence errors you can check yourself with patience and a calculator, or with a tool that recomputes them. Design and analysis errors are better reviewed by someone experienced, because they require judging whether the method answers the question.
Tell the editor as soon as possible. If the manuscript is under review you can send a correction or include it in the response. If it is already published, journals have a correction procedure. It is better to acknowledge it than to have someone else find it.
Some do, especially at journals that use automated checking tools. Even when a reviewer does not, a p value that is very different from what its statistic implies is noticeable at a glance, and it raises doubts about the rest of the figures.
Yes, and more so when it is simple: reporting errors (figures that do not agree, missing effect sizes, unmentioned assumptions) do not depend on how complex the analysis is. A review beforehand finds them before the reviewer does.