The p-value is probably the most widely used and most poorly understood statistical concept in all of psychology. Generations of researchers have learned to look for that magic number below 0.05 as a signal that their hypothesis is correct, that their effect is real, that their article deserves to be published. And yet, that interpretation is incorrect in almost every possible sense. The p-value does not tell you the probability that your hypothesis is true, it does not tell you whether your effect has practical importance, and its mechanical use as a publication criterion has generated a replicability crisis that psychology is still trying to resolve.
To understand the problem, it is worth remembering what a p-value actually is. It is the probability of obtaining a result at least as extreme as the one observed, assuming that the null hypothesis is true. Nothing more. It is not the probability that the null hypothesis is true given your result (that would be a Bayesian statement, and it would require a prior probability that the frequentist framework does not provide). Nor is it the probability that your result is a false positive, although this confusion is extraordinarily common even among experienced researchers. Gigerenzer and others have repeatedly documented how statistics professors, journal reviewers, and textbook authors systematically commit these interpretation errors.
How p < 0.05 distorts research
The problem is not merely conceptual. The use of the p-value as a binary threshold for decision-making has very concrete practical consequences on research quality. The most obvious is publication bias: studies with p < 0.05 get published, those that do not reach that threshold do not. This creates a biased scientific literature where effects appear larger and more consistent than they actually are. But there are more subtle and perhaps more harmful consequences.
When your career depends on obtaining significant results, the incentives to manipulate the data (consciously or not) are enormous. P-hacking, which includes practices such as testing multiple analyses and reporting only the one that comes out significant, selectively removing participants, adding covariates until the result changes, or redefining the dependent variable post hoc, is extraordinarily common. Simmons, Nelson, and Simonsohn demonstrated in 2011 that with perfectly legitimate-looking researcher degrees of freedom, one could demonstrate almost anything, including that listening to a Beatles song makes you younger. The article was a satire, but the practices it described were (and continue to be) real.
Another subtle distortion is what is known as the dictatorship of the asterisk. A result with p = 0.049 is considered significant and celebrated, while one with p = 0.051 is interpreted as an absence of effect. This difference of 0.002 in the p-value, which in practice is irrelevant, can determine whether an article is published or not, whether a thesis is successfully defended, whether a line of research continues or is abandoned. It is an absurd system that rewards randomness and penalizes honesty.
Alternatives and complements to the p-value
The good news is that alternatives are available, and the statistical community is promoting them with increasing vigor. Confidence intervals, although not free of interpretive problems, offer more information than a simple yes or no. A 95% confidence interval for a mean difference tells you the range of values compatible with your data, which allows you to evaluate both the statistical significance and the practical relevance of the effect. If the interval ranges from 0.01 to 0.03 points on a 100-point scale, you know that the effect, although statistically significant, is trivial in practice.
Effect sizes are another essential complement. Reporting Cohen's d, eta squared, the correlation r, or any other appropriate measure of effect size should be mandatory in any psychology article (and indeed APA guidelines have required it for years, although compliance is uneven). The effect size tells you how much things change, not just whether they change.
Bayesian statistics represent a more radical alternative. Instead of calculating the probability of the data under the null hypothesis, the Bayesian approach allows you to directly quantify the evidence for or against the hypotheses of interest. The Bayes factor, for example, tells you how many times more likely your data are under the alternative hypothesis than under the null (or vice versa). This answers the question that most researchers believe they are answering when they look at a p-value, but that they are in fact not answering at all.
Toward a more mature statistical culture
The debate over the p-value is not new, but it has gained renewed urgency following the replicability crisis. In 2016, the American Statistical Association published for the first time in its history an official statement on a statistical concept, with six principles on the correct interpretation of the p-value. In 2019, an editorial in The American Statistician signed by more than 800 statisticians called for abandoning the concept of statistical significance. These are important institutional movements that reflect an ongoing cultural shift.
In practice, my recommendation is not that you stop using p-values, but that you use them as one tool within a broader arsenal. Report confidence intervals, report effect sizes, consider Bayesian analyses when appropriate, and above all, do not make binary decisions based exclusively on whether a number is greater or less than 0.05. Statistics should not be a mechanical ritual but a tool for thinking rigorously about the uncertainty inherent in data. And that requires judgment, not just software.