The five statistical problems that get physiotherapy papers rejected

You have the manuscript written, the data clean and a significant between-group difference, and you still have not pressed submit. The trial has 22 patients, you measured each of them three times, the only person you managed to blind was the one handing out the questionnaires, and you picked the outcome scale because it was the one the group down the corridor used. The decision in front of you is not whether the analysis is correct: it probably is. It is whether what you wrote around the numbers survives a reviewer who has read twenty physiotherapy trials this year and knows exactly where to look. These five places are where they look, and not one of them is fixed by writing a better discussion.

'Significant' is not the same as 'matters to anyone'

The results paragraph says the experimental group improved 4.1 points more than the control on a 0 to 100 scale, with p = .03, and the discussion concludes the intervention is effective. The reviewer looks in the methods for the minimal clinically important difference of that scale in that population, does not find it, and writes one line: the clinical relevance of the observed difference is not established. That line on its own does not reject a paper. Together with the other four, it does.

The MCID is not decoration for the discussion: it is the ruler you read the confidence interval against. Report the between-group difference with its 95% interval —CONSORT 2025 asks for this explicitly in item 26, 'the estimated effect size and its precision'— and compare it with the scale's MCID. Three readings come out of that and all three are defensible: the whole interval above the MCID, which is a clinically relevant effect; the interval straddling it, which is compatible with both relevance and irrelevance and has to be written that way; and the whole interval below it, which is a real and clinically small difference. What is not publishable is failing to make the comparison.

Where the MCID comes from matters as much as the number itself. It has to be cited, estimated in a population resembling yours and in the same direction, because improving and worsening need not require the same number of points, and because anchor-based values —what the patient says they noticed— and distribution-based values —a fraction of the standard deviation— are not interchangeable. If no MCID has been published for your population, say so and use the measurement error as a provisional floor. What you cannot do is stay quiet and let the reader assume that 4.1 points is a lot.

Responder analysis —what proportion of each group exceeds the MCID— is a good way to translate the result into something a clinician understands, on two conditions: that it was prespecified and that it comes with its interval. Dichotomizing after seeing the data, trying cut-offs until one turns significant, is the kind of thing an experienced reviewer spots precisely because the cut-off is so well chosen.

Three measurements of the same ankle are not three ankles

Physiotherapy generates dependent data by design and almost without trying. You measure before, after and at one month. You measure both knees of the same patient. You measure strength in four muscle groups. You log ten sessions per person. Every one of those repetitions is valuable information and none of them is a new participant: the second measurement from a patient resembles their first far more than it resembles anyone else's, and that resemblance is exactly what an independent-cases analysis throws away.

What gives the problem away is not the wording, it is the degrees of freedom. If the study has 24 participants measured at three time points and the table reports an F with 69 error degrees of freedom, the story is already told: 72 observations were analyzed as if they were 72 people. The interval comes out too narrow, the p value too small and the effect too pretty. It is the easiest error to detect from outside and the most expensive to fix afterwards, because it changes every number in the paper at once.

The standard fix is a mixed model with a random intercept per participant, which acknowledges that measurements from one person travel together; with both legs, limb is nested within patient. A repeated-measures ANOVA works if the design is balanced and nothing is missing, but in physiotherapy something almost always is, and the mixed model tolerates it without deleting the whole participant. If you report a mixed model, say which covariance structure you used and how you computed the degrees of freedom: reviewers who know this will ask, and reviewers who do not were never going to raise it.

A close relative of the same problem: comparing pre with post within each group, seeing p = .01 in the intervention arm and p = .21 in the control, and concluding the intervention works. You have not shown that. What you need to test is the group by time interaction, or directly the between-group difference in change. One effect being significant and another not does not mean the two differ from each other.

The blinding you cannot do and the two you can

Nobody is going to reject your trial because you did not blind the physiotherapist delivering the manual technique. That is impossible and the guidelines know it: CONSORT 2025 item 20a asks who was blinded after assignment to interventions —'participants, care providers, outcome assessors, data analysts'—, and that is a question about who, not a demand that everyone was. Blinding, unlike concealment of the allocation sequence, cannot always be done, and on the manual therapist's side it almost never can. The PEDro scale, which is what half of physiotherapy uses to rate trial quality, splits blinding into three separate items: item 5 for subjects, item 6 for the therapists who administered the therapy and item 7 for the assessors. They are three independent things precisely because one can be missing without the others.

What does get you rejected is reporting it badly. Writing 'double-blind' about a study in which the therapist obviously knew what they were delivering tells the reviewer you copied a formula without thinking about it, and from there they read the rest with a magnifying glass. Saying nothing at all is worse, because what it reads as is that nobody was blinded, including whoever took the measurements.

The two forms of blinding almost always within reach are the assessor and the analyst. The assessor is whoever holds the goniometer or hands over the questionnaire, and they have no need to know which arm the patient is in: it is enough that they are not the person delivering treatment and that the patient is asked not to tell them. The analyst is cheaper still, because it is solved by labeling the groups A and B until the primary analysis is locked. Both fit in one line of the methods and neither costs money.

If the patient could not be blinded, say so and say which way that pushes. Subjective outcomes —pain on a visual analogue scale, perceived function, satisfaction— move most when the patient knows they got the new treatment; a range of motion measured by a blinded assessor moves far less. A limitation that names the bias, its direction and the outcomes it touches is worth considerably more than the 'results should be interpreted with caution' that appears in half of all manuscripts. And if you also asked participants which group they believed they were in, you have a piece of data instead of an excuse.

The problem is not the small sample, it is what you claim with it

CONSORT 2025 item 16a asks how the sample size was determined, 'including all assumptions supporting the sample size calculation'. Most small physiotherapy trials answer that in one of two ways: by not answering, or by writing that it was computed in a power program assuming a large effect size. The second is worse than the first, because the assumption was chosen backwards: you picked the effect you needed in order to arrive at the sample you could actually recruit.

And then there is post hoc power, which tends to appear when the result does not come out. It carries no new information: power computed from the effect you just observed is a rewriting of the p value, and by construction a non-significant result always comes back with low power. There is nothing to learn from a number you already know. The answer to 'the study appears underpowered' is not a power figure, it is the confidence interval: it says which effects your trial rules out and which it does not, which is the question underneath.

With a small sample there are three honest routes and all three hold up in front of a reviewer. The first is to declare the study a pilot or feasibility trial and then behave like one: recruitment, adherence and safety outcomes, estimates with their intervals, and no hypothesis test sold as evidence of efficacy. The second is to justify the sample by precision: with 30 patients per group the interval around the difference will be roughly this wide, and this is what it will and will not rule out. The third is the ordinary prespecified calculation, with assumptions taken from a source you cite rather than from convenience.

The multiplier on this problem is not the n, it is the outcomes. Twenty patients and fourteen outcome variables with no declared primary is a machine for finding significant results that do not replicate, and it happens a lot in physiotherapy because measuring one more variable costs two minutes of a session. Declare one primary outcome, decide before you look and treat the rest as exploratory with that word written down. How to write all of it up is in how to report a power analysis in APA 7.

The scale with no measurement error attached

You report a mean change of 5 points and call it improvement. The question the reviewer asks is how much your instrument moves when nothing is happening. If the smallest detectable change of that scale in stable patients is 9 points, the 5 you measured cannot be distinguished from noise at the individual level, and saying that patients 'improved' claims more than the instrument allows.

The COSMIN criteria for good measurement properties put the bar exactly there. For measurement error, a sufficient rating is literally 'SDC or LoA <MIC': the smallest detectable change, or the limits of agreement, below the minimal important change. And if the MIC has not been defined, the property is rated indeterminate — not good, not bad, no information. The two numbers travel together, which is why the instruments section of a well-written physiotherapy trial always carries both, with a citation for each.

There is also a difference between the reliability of the instrument and the reliability of your measurement. If three different people used the goniometer, the ICC that matters is not only the one from the validation paper: it is the one from your assessors in your sample, with its model and its interval, and it is worth reporting even when it comes out worse than expected. It nearly always comes out worse, because the validation study was run in laboratory conditions and yours was run in a busy clinic. How to specify that model is in how to report the ICC and interrater agreement, and the review of the outcome scales most used in physiotherapy, with their properties, is in outcome scales in physiotherapy.

The last distinction is the one most often muddled: the smallest detectable change is an individual threshold, and a group mean is estimated with far less error than any single patient. A mean change below the SDC can be perfectly detectable as a group effect and at the same time irrelevant for any particular patient. Both sentences are true and you have to say which one you are asserting, because if you do not, the reviewer will assume the one that suits you least.

The five lines in the letter, and what you can check first

The five problems do not arrive with their names attached. They arrive like this. 'The clinical relevance of the observed difference is not established'. That is the MCID. 'The analysis does not account for the repeated measurements within participants'. Those are the degrees of freedom. 'It is unclear who was blinded'. That is item 20a. 'The study appears underpowered'. It is almost never asking for post hoc power: it is asking you not to claim efficacy. 'The psychometric properties of the outcome measure are not reported'. That is measurement error. Translated, all five can be answered; untranslated, they look like a generic rejection.

Before you submit there is a pass that takes an afternoon and demands no more statistics than you already know. Recompute every p value from its test statistic and its degrees of freedom, and check that those degrees of freedom match the number of people rather than the number of measurements. Check every effect size against the F or the t printed next to it, which is where the inconsistencies from the time you redid the analysis and did not redo the table show up. And check that every reference exists and says what you say it says.

That pass is what the Previum example report automates: it recomputes p values from the degrees of freedom in code, checks effect sizes against the manuscript's own F or t, and verifies references one by one against eight databases. The prose of the report is written by a model, but around numbers it cannot change. It is not peer review: it is an author's self-assessment of their own manuscript before submission. What it will not do is decide for you whether 4.1 points matter, because that remains a clinical question.

Several of these problems never reach a reviewer: they end in a desk rejection, with the editor returning the physiotherapy paper in days. Before you submit, run a desk-reject check on your manuscript.

Frequently asked questions

No MCID has been published for my scale in my population. What do I write?

Write that sentence, in the methods, and then do three things. Use the instrument's smallest detectable change as a provisional floor, which at least tells you where noise ends. If you asked patients for a global impression of change, you have an anchor and can estimate an MIC in your own sample, and estimating it is a result in itself. And if you are running a responder analysis, run it with two plausible thresholds and present both: an MCID imported from another population is an assumption, and assumptions get a sensitivity analysis rather than a hiding place.

I cannot blind the physiotherapist delivering the technique. Will that get me rejected?

No. You will lose PEDro item 6, which is the same point that essentially every manual therapy trial loses, and it is not what decides a letter. What decides it is how you report it: state who was blinded and who was not, as CONSORT 2025 item 20a asks, blind the assessor and the analyst, which you almost always can, and in the limitations name the bias with its direction and the outcomes it affects. Calling a manual therapy trial 'double-blind' does more damage than blinding nobody.

I have 18 patients and the result is significant. Do I submit?

Submit it, but decide first what you are submitting it as. With 18 patients the confidence interval is wide by definition, so look at its lower bound: if that bound sits below the MCID, your own result is compatible with a clinically irrelevant effect, and you are better off writing that yourself before a reviewer does. A significant effect from a small sample is also inflated on average, because only large estimates cross the threshold. And do not add post hoc power to defend it: that makes the letter worse.

Can I treat a patient's right and left leg as two cases?

As two independent cases, no: they share a patient, a history and a therapist, and counting them separately doubles your sample without doubling your information. There are three clean routes. Pick one limb per patient, at random and stated in advance. Average the two and analyze the average. Or model limb nested within patient, which is the one that actually uses the data. If you take the third, report the intraclass correlation of the clustering: that is what justifies the model.

A reviewer is explicitly asking for post hoc power. Do I give it to them?

First answer what they actually want to know, which is what effects your study rules out: give the confidence interval around the difference and say what falls inside it and what falls outside. Explain along the way, without picking a fight, why power computed from the observed effect adds nothing: it is the same p value on another scale, so a non-significant result will always return low power. If they insist, include it with a sentence saying what it is, because this is not a battle worth having: you lose three weeks and the reviewer still holds the pen.

Will Previum tell me whether my trial meets CONSORT?

It does not fill in the CONSORT checklist for you and it does not decide whether your difference is clinically relevant. What it does is the numbers layer: it recomputes p values from the degrees of freedom, checks effect sizes against the manuscript's F or t, and verifies references against eight databases. That is where degrees of freedom that do not match the number of patients fall out. The rest —the MCID, the blinding, the sample size justification— may come up in the prose of the report, but deciding what to write is yours. You can try it in Previum, which has a free Basic version alongside the Professional one.

Keep the underlying idea: the reviewer is not hunting for arithmetic errors, they are looking for the gap between what you measured and what you claim. The MCID closes that gap from above, measurement error closes it from below, the degrees of freedom say how much information you really have, blinding says whose judgment you are trusting and the sample size justification says what you were entitled to conclude before you looked. Go through all five with the manuscript in front of you before you submit, and if by the third one it is clear the analysis has to be redone, statistical analysis for physiotherapy sets out what doing it properly involves.