Errors in the interpretation of statistical results are not exclusive to students or novice researchers. Several studies have documented that even statistics professors, journal editors, and experienced reviewers commit interpretive errors with alarming regularity. The problem is not a lack of intelligence but a lack of reflection: many erroneous interpretations have been repeated so often in articles and textbooks that they have become a kind of statistical folklore transmitted from generation to generation without being questioned.
The most widespread error -- explored in depth in the tyranny of the p-value -- is the inversion of the conditional in the interpretation of the p-value. The p-value is the probability of obtaining the observed data (or more extreme data) assuming the null hypothesis is true: P(data | H0). Many researchers interpret it as the probability that the null hypothesis is true given the data: P(H0 | data). The difference between the two is fundamental and is not a technicality. P(rain | clouds) is very different from P(clouds | rain). The fact that the data are unlikely under the null does not mean that the null is unlikely given the data. This confusion, documented by Cohen, Gigerenzer, and many others, lies at the root of many unfounded conclusions in the psychological literature.
Confusing statistical significance with practical importance
Another frequent error is treating statistical significance as synonymous with importance or relevance. A result can be statistically significant and practically irrelevant. With a sample of 10,000 participants, you can detect as significant a difference of half a point on a 100-point scale. That difference is real in the statistical sense (it is probably not due to chance), but it is so small that it lacks any clinical or practical relevance. The antidote to this error is to always report and interpret effect sizes alongside p-values (our APA results formatter makes this step easier). Cohen's d, partial eta squared, the odds ratio, or any other measure of effect size tells you how much things change, which is the question that truly matters.
The inverse error is also frequent: concluding that there is no effect when the result is not significant. A p-value of 0.15 does not prove that there is no effect; it only indicates that the data do not provide sufficient evidence to reject the null at the conventional criterion. The difference is subtle but important. With a small sample, it is perfectly possible to have a real effect of moderate size and not detect it as significant, simply because the statistical power is insufficient. This is why it is crucial to interpret non-significant results in the context of the study's power and the confidence interval: a narrow confidence interval that includes zero is more informative than a wide interval that also includes it.
Almost all of these errors are committed while reading somebody else's result: a discussion paragraph that claims more than its numbers support, or one you do not follow well enough to tell whether it is overclaiming. When that happens, paste the paragraph into explain this paragraph: it comes back in plain language with a glossary of the technical terms, which is exactly what you need to decide whether to cite it or argue with it.
Errors with correlations and regression
The causal interpretation of correlations is probably the error most mentioned in statistics courses and the one on which that mention has the least effect. Researchers continue to use causal language to describe correlational associations. Part of the problem is that everyday language is inherently causal: we say that stress "produces" anxiety, that self-esteem "improves" performance, that poverty "causes" mental health problems. Translating these statements into purely associative language requires a conscious effort that many researchers do not make, especially in discussion sections where the temptation to over-interpret is greatest.
In multiple regression, a common error is interpreting the coefficients as independent effects of each predictor, ignoring the problem of multicollinearity. When two predictors are highly correlated, their regression coefficients become unstable and difficult to interpret in isolation. A predictor may have a non-significant coefficient not because it is unrelated to the dependent variable, but because its shared variance with another predictor is already captured by that other predictor. Examining the correlation matrix and variance inflation factors (VIF) before interpreting the coefficients should be a mandatory step in any regression analysis.
The post-hoc power analysis fallacy
One of the most persistent errors in the literature, and one that reviewers themselves sometimes request, is the calculation of "observed" or post-hoc power: that is, computing the statistical power that the study had given the effect size observed in the sample. This procedure is mathematically uninformative and conceptually wrong, as Hoenig and Heisey (2001) demonstrated definitively. Observed power is a monotonic function of the p-value: if p is less than 0.05, observed power is automatically greater than 50%; if p is just above 0.05, observed power will be just below 50%. It therefore adds no information beyond what the p-value already provides. Worse still, when a result is not significant, observed power is necessarily low, and that low power is then used to "explain" the lack of significance, which is circular. The correct power analysis is the one done a priori, with the minimum effect size of clinical or theoretical interest as input, not the observed effect. For more depth on this topic, see my article on post-hoc power analysis.
Dichotomizing continuous variables: a hidden but expensive error
Another recurring practice with greater consequences than usually recognized is the dichotomization of continuous variables. Authors take a continuous score (depression on the BDI, anxiety on the STAI, age, IQ) and convert it into a binary variable using the median or a clinical cutoff. This is justified as "facilitating interpretation" or "creating clinically meaningful groups." MacCallum, Zhang, Preacher, and Rucker (2002) showed that median splits reduce the statistical power of the analysis equivalently to discarding approximately one-third of the sample, distort the estimated effects, and can even invert the sign of true associations when there are interactions. The robust alternative is to maintain the variable as continuous and, if interpretation in terms of "high" and "low" is desired, to report estimated effects at the mean plus or minus one standard deviation, which preserves all the information without distorting the analysis. Dichotomization may seem more "intuitive," but the cost in precision and validity is too high to justify it except in cases where the underlying construct is genuinely categorical.
Simpson's paradox: when subgroups tell the opposite story
Simpson's paradox is one of the most fascinating phenomena in statistics and one of the easiest to overlook. It occurs when the direction of a relationship between two variables changes when an aggregate sample is broken down into subgroups, or vice versa. The classic example is the case of admission to UC Berkeley in 1973: in the aggregate, the admission rate for women was lower than for men, suggesting gender discrimination; but when broken down by department, women had equal or higher admission rates in most departments. The explanation was that women applied to more competitive departments. In psychological research, Simpson's paradox appears frequently when there are unobserved or unmodeled subgroup variables: an intervention can appear effective in the aggregate but be harmful in a specific subgroup, or vice versa. The implication is clear: always inspect your data at multiple levels of aggregation, conduct moderation analyses on theoretically relevant variables, and beware of "main effects" without exploring potential interactions.
Ignoring effect sizes systematically
Although it is now mentioned in every methods textbook, the systematic omission of effect sizes remains an endemic problem. The published evidence shows that even in journals that explicitly require effect sizes, between 30% and 50% of articles either fail to report them or report them incorrectly. The most common errors are: reporting eta squared without specifying whether it is partial, generalized, or classic (which can lead to differences of an order of magnitude); reporting Cohen's d without specifying which standard deviation was used in the denominator (Cohen's d, Hedges' g, Glass's delta differ in this); failing to report confidence intervals for effect sizes (which is now mandatory in APA 7); and using interpretive rules of thumb (small, medium, large) outside the original context. Cohen himself warned that his thresholds were tentative and field-dependent. In psychology, a d of 0.20 between an active intervention and a control can be clinically meaningful in chronic conditions, while in basic experimental research the same value can be trivial. Always contextualize the effect size with the existing literature in your specific area.
Other common errors that I see less mentioned but that have an impact
There are errors that, although less mentioned, appear with worrying frequency in manuscripts that come to consultation. The first is the comparison of correlations without a formal test of difference: if you have r equals 0.45 in one group and 0.30 in another, you cannot conclude that they differ without applying a Fisher's z transformation test or, better, formally modeling moderation. The second is multiple regression with collinear predictors interpreted as if the coefficients were independent effects: I have already mentioned this, but it deserves repetition because it is the most frequent interpretive error in psychology articles using regression. The third is the use of indices such as R squared in nested models without considering shrinkage: with small samples and many predictors, R squared is highly inflated, and the adjusted R squared corrects only partially. The fourth is the interpretation of significant interaction effects without inspecting simple slopes or the Johnson-Neyman region of significance: knowing that there is interaction is not enough; you have to characterize where and how the effect changes.
Recommendations for avoiding these errors
The best protection against interpretive errors is constant self-criticism. Before writing a conclusion, ask yourself whether your data truly support it or whether you are extrapolating beyond what the design and analysis allow. Read your results section as if you were a skeptical reviewer and identify the points where you might be over-interpreting. And do not be afraid to acknowledge uncertainty: an article that states "our data are consistent with the hypothesis that X influences Y, although the cross-sectional design does not allow confirmation of the causal direction" is more honest and more credible than one that categorically asserts that X causes Y based on a correlation. If you need guidance with your analyses, our doctoral thesis analysis service can help you avoid these errors from the design stage.