The five statistical problems that get clinical medicine papers rejected

You have a manuscript nearly finished, or you have a letter in front of you with two reviewers saying different things, and the question is the same either way: what has to be touched before someone with decision-making power reads it. In clinical medicine that decision has a particular flavor. Outcomes are binary or time-to-event, patients get lost along the way, and the final reader is not a methodologist. It is someone who wants to know whether treating their patients this way changes anything they would notice in clinic. That is where most of the papers that fall on statistics actually fall, and not because of anything difficult. They fall on five recurring things, and what follows is what the reviewer looks at in each, and which of them you can still close yourself.

The reviewer is not hunting for sophistication, only for what is missing

The person your editor sends the statistical part to does not start with the model. They start with the count: how many patients you recruited, how many enter each analysis, how many are missing and why. Then they look at which measure of association you used and whether it fits the design you actually have, and only then do they reach the model and what that model takes for granted. It is a short route, done with the flow diagram and the results table open, and it ends with an impression that does not shift much through the rest of the reading.

The questions are no secret, because they are published. CONSORT 2025 devotes item 21c to how missing data were handled in the analysis and item 21d to the methods for any additional analyses, subgroup and sensitivity analyses included, distinguishing prespecified from post hoc; its item 26 asks, for binary outcomes, for the absolute effect size as well as the relative one. STROBE, for observational studies, asks in item 12(b) for the methods used to examine subgroups and interactions, in 12(c) for how missing data were addressed, in 14(b) for the number of participants with missing data for each variable of interest, and in 16(c) that you consider translating relative risk estimates into absolute risk for a clinically meaningful period (checked in September 2026).

That list is half a review done before it starts. If your manuscript answers those items where the reviewer goes looking for them, the conversation becomes a conversation about your results. If it does not, the conversation is about why it does not, and that one you do not win even when you are right.

The odds ratio read as if it were a risk

The typical case is not an arithmetic error. It is a cohort with a frequent outcome (a postoperative complication, 30-day readmission, treatment response), a logistic regression because that is what the menu offers, an OR = 2.67, and a discussion saying «exposed patients were 2.67 times more likely to be readmitted». That sentence describes a risk ratio. What you have are odds, and the more frequent the outcome, the further the two measures drift apart, always in the same direction: the odds ratio exaggerates.

This has been counted. Holcomb and colleagues (2001) reviewed the articles using odds ratios published between 1998 and 1999 in Obstetrics & Gynecology and the American Journal of Obstetrics and Gynecology. Of 151 studies, 107 allowed a risk ratio to be estimated as well, and the difference between the two measures exceeded 20% in 47 of those 107 (44%). In 39 of the 151 articles (26%) the odds ratio was interpreted as a risk ratio without explicit justification. That is two journals and one field, so do not carry the figure across as a general rate; the mechanism, however, does travel, because it depends only on how frequent the outcome is.

The way out is not to change a word in the discussion, it is to choose the measure your design permits. In a case-control study sampling is done on the outcome, risks are not estimable, and the odds ratio is the correct measure. In a cohort or a trial with follow-up you can estimate the risk ratio or the risk difference directly with log-binomial regression or Poisson regression with robust variance, and then there is nothing to approximate. And if you keep the odds ratio, say odds when you mean odds. Reporting the absolute difference as well saves you half the comments, and it is precisely what CONSORT 2025 asks for in item 26 and what STROBE suggests in 16(c). The full comparison between odds ratio and risk ratio, including when they approximate each other and when they do not, has its own article.

The reviewer's comment almost never says «you have confused OR and RR». It says the interpretation is not supported by the measure presented, or asks you to temper the conclusions. Translated into work: rewrite the discussion, sometimes redo the model, and always revisit the abstract, which is where that sentence tends to be at its boldest and where most people will read it.

A Cox model whose proportional hazards nobody checked

The Cox model returns a single hazard ratio for the whole follow-up, and that is exactly what it assumes from the start: that the ratio between the two groups' instantaneous hazards stays constant over time. If the Kaplan-Meier curves cross, separate late or converge at the end, that single number is an average of things pulling in different directions. An HR = 0.94 can be a large effect over the first six months and nothing afterwards, and in clinical practice those two stories are not remotely the same story.

That almost nobody checks it has also been measured. Jachno, Heritier and Wolfe (2019) reviewed 66 phase III trials with a time-to-event primary outcome published between January and June 2017 in the New England Journal of Medicine, the BMJ, The Lancet and the Journal of Clinical Oncology: 98% presented Kaplan-Meier curves, 88% the log-rank test and 97% a Cox model, but of the trials reporting inference from the Cox model, only 11% reported any results of testing the proportional hazards assumption. Being in the majority protects you from nothing here: the reviewer who asks does not care about the other 89%.

Checking it is three lines of code and a paragraph. Schoenfeld residuals, global and per covariate, a plot of log(-log(S)) against time and an interaction term between the covariate and time cover what is expected. Write in the Methods what you did and what came out, even when it comes out clean, because that sentence saves an entire review round. The route from Kaplan-Meier to Cox, with censoring, the log-rank test and the assumption itself, is worked through step by step with code.

And if the assumption does not hold, nothing gets thrown away, you change what you report. A period-specific hazard ratio, survival at fixed time points (at 12 and 24 months), restricted mean survival time up to a clinical horizon, or stratification on the offending covariate are all accepted answers, and usually more useful to the clinician than a single HR. Watch the opposite trap too: a non-significant Schoenfeld test with 60 events does not demonstrate proportionality, it only says you lack the power to detect the alternative. Say it that way and you gain credibility instead of losing it.

The subgroups that appeared after someone looked at the data

The primary result comes out modest, or does not come out at all, and at some point somebody looks by age, by sex, by baseline severity, and up pops p = .03 in the over-65s. The temptation is obvious and so is what follows: the paper changes its story and the title starts talking about a subgroup. The clinical reviewer has seen that film many times, which is why they read that section with more suspicion than any other part of the manuscript.

They have reasons. Sun and colleagues (2012) reviewed randomized controlled trials published in 2007 in the core clinical journals, as defined by the National Library of Medicine, in Medline: of 207 reporting subgroup analyses, 64 (31%) made claims of a subgroup effect for the primary outcome. Of those 64, only 26 (41%) had clearly prespecified their hypotheses and only 6 (9%) had carried out a test of interaction that proved statistically significant. Fifty-four of those 64 claims (84%) met four or fewer of the ten credibility criteria the authors applied.

The technical point is the one most often skipped: comparing the p value in one subgroup with the p value in the other is not the test. Getting p = .03 in the over-65s and p = .21 in the younger group does not establish that the effect differs between them; that is what the interaction term is for, and it tends to look considerably less impressive. Then multiplicity is added on top: with six subgroup variables, finding one significant by chance stops being a surprise and becomes the expected outcome.

None of this forbids exploring. What sinks the manuscript is not having exploratory subgroups, it is presenting them as confirmatory. State what was in the protocol and what was not, which is literally what CONSORT 2025 asks for in items 21d and 28; give the interaction test with its interval; do not move the paper's main message on the back of a post hoc finding; and write in the discussion that it is hypothesis-generating. A reviewer accepts an honest exploratory finding. What they do not forgive is the feeling that something was being slipped past them.

The patients who are missing and appear nowhere

The Methods say n = 214. In Table 2 the percentages are calculated on 187. Nowhere does it say what happened to the other 27, and the flow diagram, if there is one, stops at randomization. It is the easiest of the five to find, because you do not need to understand the study to see it: subtraction is enough.

Bell and colleagues (2014) reviewed trials published between July and December 2013 in the BMJ, JAMA, The Lancet and the New England Journal of Medicine: of 77 eligible articles, 73 (95%) had some missing outcome data, with a median of 9% of participants missing. The most common method in the primary analysis was complete case analysis (45%), and only 35% reported any sensitivity analysis. Missing data is the norm; what separates one manuscript from another is whether it says so.

Analyzing complete cases only is not «doing nothing» about the missing: it is a statistical decision that assumes the people who left are interchangeable with the people who stayed. In clinical research that is rarely credible, because patients drop out for reasons tied to the outcome: they deteriorate, they have adverse events, or they improve and stop attending. The reviewer who sees this knows your effect may be biased in a direction you can reason about and they cannot, and that is where the requests start.

The minimum that closes the comment fits in half a page: how many are missing and on which variable (STROBE, item 14(b)), why they were lost, the baseline comparison between completers and non-completers, the method you used to handle them (CONSORT 2025, item 21c), and a sensitivity analysis showing the conclusion does not depend on that method. Not one of those five things needs another patient. How that half page is written, in what order and with which sentences, is in the missing data guide.

Significance told as if it were clinical relevance

With 4,000 patients almost everything comes out significant. A difference of 1.8 mmHg in systolic blood pressure, of 0.4 points on a 0 to 10 scale, or of two days in mean length of stay can carry an immaculate p < .001 and change no clinical decision at all. When the discussion calls that a «clinically relevant improvement», the reviewer is not arguing with your statistics: they are pointing out that you have fused two different questions into one sentence.

The American Statistical Association put it in writing in 2016, in the fifth of its six principles on p values: a p value, or statistical significance, does not measure the size of an effect or the importance of a result. And in the third, that scientific conclusions should not be based only on whether a p value passes a specific threshold. The ICMJE recommendations, followed by many medical journals, point the same way: do not rely solely on hypothesis testing, quantify findings with indicators of uncertainty such as confidence intervals, and distinguish prespecified from exploratory analyses (checked in September 2026).

What convinces a clinical reviewer fits into three numbers. The absolute difference between groups with its confidence interval, the number needed to treat when the outcome is binary, and an explicit comparison with the minimal clinically important difference for the instrument you used. If that threshold exists for your scale, cite it and place it next to your interval; if it does not exist for your population, say so and argue from the hard outcome instead. It is a well-built discussion paragraph, not a new analysis, and it changes how everything else reads.

The mirror-image error costs just as much: reading «not significant» as «no effect» when the confidence interval runs from a benefit that would change practice to a small harm. That is not a negative result, it is an inconclusive study, and saying it that way is both more defensible and more useful to whoever reads you. What to do when the result is not significant works it through with the exact sentences for the discussion.

Which ones you can fix with the data you already have

Sort them by what it costs to close them, which is almost the reverse of how frightening they look. Missing data and the proportionality check are reanalysis with what is already on your machine: one well-run afternoon. The choice between odds ratio and risk ratio is either reanalysis or pure rewriting, depending on your design. Significance told as relevance is only rewriting, and it moves the reviewer's impression more than anything else per hour spent. The single one that sometimes has no fix is the subgroup: if it was not planned, no analysis turns it confirmatory, and the job is to demote it in your own text and live with that.

The practical order is this. First the numbers reconcile: counts, denominators, degrees of freedom for every test, and the table saying the same thing as the text. Then you decide the measure that matches the design. Then you check the model's assumptions. And only at the end do you write the discussion, which is where three of the five errors on this list accumulate: the odds ratio, the subgroup and the relevance claim. Writing the discussion before the numbers are settled is the fastest way to write it twice.

Part of that first pass is arithmetic and can be automated: recomputing p values from the degrees of freedom, checking effect sizes against the F or the t you report yourself, and verifying references one by one against the databases. That is what Previum does in code before any model writes a sentence, and the example report shows exactly what comes out and in what format. What no machine does is decide whether your odds ratio fits your design or whether those 1.8 mmHg matter in your population. That is clinical judgment, and it means reading the whole study.

Several of these problems never reach a reviewer: they end in a desk rejection, with the editor returning the medical paper in days. Before you submit, run a desk-reject check on your manuscript.

Frequently asked questions

My study is a case-control and the reviewer wants the risk ratio. Do I give it to them?

You cannot, and that is the answer to write. In a case-control study sampling is done on the outcome, so the risk in each group is not estimable and the odds ratio is the correct measure. Answer with the technical sentence (the design does not allow absolute risks to be estimated) and offer what you can: the OR with its interval and, if you have an external estimate of outcome prevalence, a comment on how far it would sit from the risk ratio. A reasonable reviewer accepts that. What they do not accept is an RR computed as though you had a cohort.

The Schoenfeld test is significant. Do I throw the Cox model out?

No. You look at where the assumption breaks and change what you report. If the effect concentrates in the first months, a period-specific HR shows it better than a global one. If the curves converge, restricted mean survival time up to a clinical horizon is more interpretable and does not require proportionality. And if the offender is a confounder that is merely in the way, stratifying on it settles the case. Report the diagnosis and then the decision, in that order: for the reviewer the problem is never that the assumption fails, it is finding out themselves.

The reviewer wants a subgroup analysis I never planned. Do I run it?

Yes, and you label it. An analysis requested at review is post hoc by definition, so it goes with its interaction test, with a multiplicity caveat if there are several, and with a sentence saying it was done at the reviewer's request and is hypothesis-generating. What must not happen is that subgroup climbing into the abstract or changing the main conclusion. Labeled properly, the reviewer usually settles; labeled badly, it becomes the reason for the next rejection.

How much missing data is too much?

There is no threshold, and be wary of anyone who gives you one. Twenty per cent missing at random is less of a problem than 5% missing because the sickest patients dropped out. What gets examined is the mechanism, not the percentage: whether those missing resemble those remaining on baseline variables, whether the loss is related to the outcome, and whether the conclusion survives a sensitivity analysis. Show those three things and the percentage stops being the question.

How do I say my significant result is not clinically relevant without sinking the paper?

By saying it before the reviewer does, which is when it adds instead of subtracting. Give the absolute difference with its interval, compare it with the relevance threshold for your instrument, and write what it means: that the effect exists and is small, that it is consistent with what is published, or that it may matter at population level even if it does not change an individual decision. A paper that measures a small effect well and says so gets published. A paper that inflates a small effect falls, and falls in the second round.

Is it worth fixing this before submitting, or should I wait and see if they ask?

Before, and not out of perfectionism: because of the clock. These five rarely arrive as a minor revision. They arrive as major revision or as a rejection with an invitation to resubmit, and each round is two to four months. Closing them before submission is a week of work with the data you already have. Closing them afterwards is the same week, plus the wait, plus an editor who has already read you once with an impression formed.

Keep the order: counts and denominators first, then the measure your design supports, then the model's assumptions, and the discussion last, because that is where the interpretation errors all become visible at once. Four of the five close with the data you already have; the fifth closes by demoting the subgroup to exploratory and saying so out loud. And if the one you have landed on is the second or the third, and learning survival analysis the week before submission does not appeal, that is what statistical analysis for medicine is there for.