In short
Statistical rejections in sports science repeat themselves, and there are not many of them. The first one is historical: magnitude-based inference (MBI), which Medicine & Science in Sports & Exercise stopped accepting in 2018, after a reanalysis published in that same journal showed that at precisely the sample sizes its proponents called optimal the type I error rate rises to two to six times that of standard hypothesis testing (Sainani, 2018). The other four are structural and still fully intact: samples of ten or twelve athletes carrying claims sized for three hundred, repeated measures analyzed as if every repetition were a new participant, a minimal detectable change nobody has computed from their own data, and a panel of twenty performance variables from which three are reported. Two of the five can be caught in twenty minutes because they are pure arithmetic: the degrees of freedom have to match the design, and each effect size has to match its own F. The problem is almost never that the sample is small, but the distance between what that sample supports and what the abstract says.
You have a finished manuscript and an eight-week protocol with fourteen players that took you two seasons to set up. The decision in front of you is not about writing: it is which journal you send it to, with which analysis, and what you are prepared to claim from that sample. You are assuming the reviewer will go after the n, and they probably will, but that is not where most performance manuscripts fall apart. They fall apart in four rather duller places, and in one that never even reaches a reviewer, because the editor stops it first. Here they are, in the order worth checking them.
Magnitude-based inference: the rejection that never reaches a reviewer
MBI appeared in 2006 as an answer to a real problem in this field. With twelve cyclists, standard testing almost never crosses the threshold, and writing «no significant differences were found» when performance rises by 2 % in a final is a fairly poor way of describing what happened. MBI proposed something else: set the smallest worthwhile change in advance, then classify the observed effect as likely beneficial, possibly trivial or harmful, depending on which part of the interval fell on each side of that threshold. The underlying intuition is good and still is: in performance research, magnitude matters more than a cut-off at .05. The problem was never the intuition, it was the machinery underneath it.
In 2018 Kristin Sainani reanalyzed the method using the simulation code of its own proponents and published the result in Medicine & Science in Sports & Exercise: at the small-to-moderate sample sizes MBI's proponents called optimal, the method lowers type II error at the cost of inflating type I error to two to six times that of standard hypothesis testing, and that rate varies unpredictably with the sample size and the chosen smallest important effect (Sainani, 2018). The journal's editor subsequently announced that manuscripts using MBI would no longer be accepted, and this is not corridor gossip: the method's own authors report it while replying to the critique (Hopkins & Batterham, 2018). A second critique followed in 2019 in Scandinavian Journal of Medicine & Science in Sports, arguing that MBI is neither Bayesian nor a valid method of inference (Sainani et al., 2019), and that same year Hopkins rebranded the procedure as magnitude-based decisions after consulting the statistician Sander Greenland, who objected to the word inference unless it covered, beyond sampling uncertainty, the biases that come from violating the sampling and model assumptions.
How much published material is affected was measured by a systematic review in PLOS ONE: 232 studies using MBI, with a median of 10 participants per group (interquartile range 8 to 15) in multi-group designs and 14 in total in single-group designs, and with cases where effects associated with p values around .2 or .3 were declared «likely» (Lohse et al., 2020). The same authors note that several journals in the field now either caution against MBI or have stopped accepting it altogether. Individual policies change and this was checked in September 2026, so if this is your main analysis, look up the target journal's policy before you submit anything: it is the only one of the five problems on this list that can send your package back without a reviewer ever reading it.
Redoing the analysis does not mean throwing the study away. The data, the design and the smallest worthwhile change you defined are all still valid, and that threshold is probably the best methodological decision in the manuscript. What changes is how the interval turns into a sentence: report the effect with its confidence interval, state explicitly which part of that interval lies beyond the smallest worthwhile change, and stop converting that into a probability that the effect is beneficial, which is exactly the step that does not hold. And do not do the obvious thing, which is swapping «inference» for «decisions» in the analysis section and resubmitting: the rename is published and has been common knowledge in the field since 2019.
Ten elite athletes: the problem is not the sample, it is the sentence
Nobody reviewing in this field will ask you for two hundred international-level rowers, because two hundred are not available. What they will ask is that the claim fit inside the sample. «Supplementation improved performance by 3 %» with twelve participants and no interval is the sentence most often underlined in a review report, because that 3 % comes with an interval that may run from 0.1 % to 6 %, and that is where the information sits: not in the fact that it went up, but in how badly you know it.
There is a side effect of tiny samples worth saying yourself before the reviewer says it. When n is very small, only large estimates cross any threshold, so what ends up published is biased upwards by construction. Your d of 1.20 will probably not replicate, and writing that in the limitations in those words usually saves you a whole round. It is one of the rare occasions where admitting a weakness raises the assessment of the manuscript, because it shows you understand what you are holding.
Then there is the post hoc power request, sometimes from the reviewer themselves. Computing it from the observed effect adds nothing, because it is a function of the p value you already reported: if the result was not significant, power computed that way will come out low by definition, and that is not a finding (Hoenig & Heisey, 2001). What does help is a sensitivity analysis, that is, the smallest effect your design could have detected at 80 % power, plus a written sample size justification, which can perfectly well rest on the real availability of athletes at that level (Lakens, 2022). How each one is written up is in how to report a power analysis in APA 7.
And then there are the design decisions that genuinely buy power. A crossover with a washout turns each athlete into their own control and removes between-subject variability, which in elite samples is enormous. Taking three measurements per session and averaging reduces instrument noise, although it does not increase n: those are two different things and it is best not to sell them as one. And adjusting for baseline with ANCOVA rather than comparing raw change scores is almost always more powerful and controls regression to the mean (Senn, 2006).
Repeated measures: when 168 jumps become 168 participants
Fourteen players, three jumps per session, four sessions: 168 rows in the spreadsheet. If those 168 rows enter the analysis as independent observations, the degrees of freedom balloon, the standard errors shrink and out come beautiful p values that mean nothing. This is pseudoreplication, and it is the easiest error in the whole manuscript to spot, because the degrees of freedom you report give it away without anyone needing to see your data: a reviewer with the design in front of them does the arithmetic in thirty seconds.
With a balanced design and no dropout, repeated-measures ANOVA is fine, provided you report Mauchly's sphericity test and, when it fails, the correction applied together with the corrected degrees of freedom rather than the original ones. The problem is that in sport the design is almost never balanced: there are injuries, call-ups, travel and a couple of athletes who miss the post-test. There, a mixed model with a random intercept per athlete is the better option, because it uses all the available information instead of deleting the whole participant, and it also allows a random slope if the training response varies between athletes, which is usually exactly what you are trying to show. The reporting conventions for both are in how to report an ANOVA in APA 7.
The third detail in this block is which variable you analyze. Comparing change scores between groups looks like the natural move and is the one that drags most regression to the mean with it, especially when the experimental group started off worse, which with fourteen participants happens half the time by pure chance. A baseline covariate in an ANCOVA solves that. And do not test baseline differences between groups with a t test in a randomized study: randomization already guarantees that any difference is chance, so that test answers no question and does draw attention to an imbalance you then fail to adjust for.
Typical error and minimal detectable change: two numbers you cannot copy from another paper
Typical error of measurement is the variability your test shows when the athlete has not changed, and it comes from a test-retest with your participants and your instrument (Hopkins, 2000). The minimal detectable change is built on top of it: at 95 % confidence, MDC = 1.96 × √2 × SEM, which is roughly 2.77 times the standard error of measurement (Weir, 2005). Below that figure you cannot distinguish real change from instrument noise in an individual athlete. It is a short calculation and its absence is conspicuous, because the entire individual-level results section depends on it.
The two usual failures here travel together. The first is borrowing reliability from another paper: the ICC depends on the between-subject variance of the sample it was computed in, so an ICC of .95 obtained in a heterogeneous sample of amateurs can drop to .60 in your elite squad with the very same device, without anything having got worse. What changed is the denominator, not the instrument, and this is developed in how to report the ICC and interrater agreement. The second failure is using the minimal detectable change as a threshold for the group mean: the standard error of a mean is far smaller than that of an individual measurement, so a mean change below the MDC can be perfectly detectable at group level. They are two different levels and mixing them shows immediately.
Alongside typical error sits the smallest worthwhile change, conventionally set in this field at 0.2 between-athlete standard deviations. Comparing the two is the most useful piece of information your methods section can carry: if your typical error is larger than the smallest worthwhile change, your test cannot detect the smallest change that would be worth having, and that has to be written down, not buried. A reviewer at a performance journal asks for both numbers side by side precisely because together they define what the study could ever have seen.
Responders and non-responders: the table that rarely survives
The table that sorts athletes into responders and non-responders according to whether their change exceeds a threshold is popular, intuitive and usually the first thing to fall. It fails in two places at once: the chosen threshold is almost always below the minimal detectable change, so you are classifying noise, and there is no comparison group with which to separate genuine individual response from the variability the same athlete would show doing nothing.
The formal argument is in Atkinson and Batterham (2015): the standard deviation of change scores in the intervention arm mixes true individual response with measurement error and within-subject biological variation, and separating them requires a control arm measured at the same time points. The standard deviation of the individual response is then estimated as the square root of the difference between the variance of change in the intervention arm and the variance of change in the control arm, and it can come out negative. When it does come out negative, the answer is that there is no evidence of differential individual response, and publishing that is a perfectly informative result.
What does survive review is the descriptive version. Plot the individual trajectories, draw the minimal detectable change band around each athlete's baseline, and call what you see «changes beyond measurement error», not «responders». The difference between those two phrases is the difference between describing and inferring a mechanism, and a reviewer who knows the field accepts the first and rejects the second.
Twenty performance variables and three p values in the abstract
A session with GPS units, a force plate and sleep monitoring returns between twenty and sixty variables per athlete per day. Reporting the three that came out and leaving the rest aside is the most common form of publication bias inside a manuscript, and it is not always deliberate: when the export has forty columns, looking at all of them and keeping the interesting ones feels like exploratory analysis and is uncontrolled multiplicity. With fifteen variables and a correlation matrix you have 105 tests, about five of which will come out significant by chance at α = .05. If the discussion rests on two of them, the reviewer will say so in those words.
The way out is not to Bonferroni-correct everything, which with forty mutually correlated metrics leaves nothing standing and is needlessly conservative when the variables measure the same thing three ways. What is being asked for is simpler: declare a primary outcome before looking at the data, define explicitly which tests form a family, and apply false discovery rate control within each family (Benjamini & Hochberg, 1995) for the exploratory block. A preregistration does that work on its own, and in sport it is still uncommon enough to be a cheap advantage over the manuscript next to yours.
The complement is transparency: a supplementary table with every variable measured, with corrected and uncorrected p values, and one sentence in the methods stating how many outcomes were recorded in total. It stops looking like you picked and starts looking like you explored, which is what you did. That table costs half an hour; leaving it out usually costs a review round asking for exactly that.
The twenty minutes before you submit
The order that saves the most time is this. First, whether the inferential framework is still acceptable at the target journal, because that decides whether the manuscript reaches review at all. Second, whether the degrees of freedom of each test match the design described in the methods. Third, whether each effect size matches the F or the t you reported yourself two lines earlier. Fourth, whether there is a typical error and a minimal detectable change computed from your own data. And fifth, how many outcomes were measured against how many are reported.
The second and third are arithmetic, not judgment, which is why they can be recomputed without anyone offering an opinion. That is what the Previum sample report does: it recomputes p values from the degrees of freedom with code, checks effect sizes against the author's own F or t, and verifies references one by one against eight databases. The prose of the report is written by a model, but around numbers it cannot change. It is worth saying what it is not: it is not peer review, it is the author's self-assessment of their own manuscript before submitting it.
The first, fourth and fifth points are not arithmetic, they are decisions, and they are taken before you analyze anything. No tool will decide for you whether your design supports the causal claim in the abstract, or whether a mixed model beats ANCOVA given your dropout, and none will compute a minimal detectable change from a test-retest you never ran. If you have got this far recognizing your own manuscript in three of the five sections, the problem is not the writing, and the wider picture of why these journals reject is in why you get rejected from Q1 journals.
Several of these problems never reach a reviewer: they end in a desk rejection, with the editor returning the sports science paper in days. Before you submit, run a desk-reject check on your manuscript.
Frequently asked questions
My study is analyzed with MBI and it is already written. Do I bin it?
No. The data, the design and the smallest worthwhile change you defined all still hold, and that threshold is the best thing in the manuscript. What has to be redone is how the result turns into a sentence: effect with confidence interval, and an explicit description of which part of the interval lies beyond the threshold, without converting it into a probability of benefit. Before resubmitting, check the target journal's policy, and do not simply rename the method magnitude-based decisions, because the rename dates from 2019 and editorial offices know about it.
Can you publish in a Q1 journal with fourteen participants?
Yes, and it happens constantly in this field. What changes with fourteen is not whether you publish, it is what you claim: confidence intervals next to every effect, a crossover or within-subject design whenever the protocol allows it, a sensitivity analysis instead of post hoc power, generalization restricted to the population you actually studied, and an explicit limitation about upward bias in the estimate. A manuscript with n = 14 and calibrated claims holds up in review; one with n = 14 and an abstract written as though there were two hundred does not.
The reviewer is asking for post hoc power. Do I compute it?
You answer them, but not with that. Power computed from the observed effect is a transformation of the p value already sitting in your table, so it adds no information about whether the study was adequately sized (Hoenig & Heisey, 2001). What you send instead is a sensitivity analysis, with the minimum detectable effect for your n at 80 % power, plus the confidence interval of the observed effect, which is what actually answers the question the reviewer is trying to ask. Explaining that in two polite sentences and offering the alternative number works better than refusing.
I have pre, post and follow-up with two dropouts. Repeated-measures ANOVA or a mixed model?
Mixed model. With dropout, repeated-measures ANOVA deletes the whole participant across all three time points, and with fourteen participants losing two means losing 14 % of the sample over one empty cell. The mixed model uses the observations you do have, handles unequal measurement timings and allows a random slope if the response varies between athletes. If the design were complete and balanced, both would give practically the same answer and you could stay with the ANOVA for reader familiarity, reporting the sphericity correction.
Do I use the minimal detectable change or the smallest worthwhile change as my threshold?
Both, because they answer different questions. The minimal detectable change tells you what change you can distinguish from measurement error in an individual athlete: it is a limit of the instrument. The smallest worthwhile change tells you what change the coach would care about: it is a judgment of relevance, conventionally set at 0.2 between-athlete standard deviations. Report both in the methods and comment on the relationship between them, because if the typical error exceeds the smallest worthwhile change, that is a first-order limitation of the study.
All my variables come from the same test. Do I still have to correct for multiple comparisons?
Coming from the same device does not make them one variable. What decides is the structure of your questions: declare a primary outcome, group the rest into families by hypothesis, and apply false discovery rate control within each family. Bonferroni across forty correlated metrics is excessive and erases real findings; correcting nothing turns the study into a hypothesis generator, and that has to be said in those words. Adding the full table with corrected and uncorrected p values closes the objection before it is raised.
Which of the five does your manuscript have?
All five are checked with the manuscript open and the design next to it, and two or three usually turn up together, because they come from the same place: a sample that is the one you could get, and claims written without looking at it. Rearranging that is analysis work, not writing work, and in training studies, functional test validation and team data it is what I do.
See how I handle performance studies →Keep the order: first check that the inferential framework is still alive at the target journal, because that rejection arrives before the reviewer does; then reconcile the degrees of freedom with the design and each effect size with its own F; and leave for last the three decisions that are not arithmetic, which are how much you claim from your sample, what your test actually measures, and how many outcomes you looked at before writing the abstract. If that list comes back three out of five and you are not sure where to start, that is exactly what my statistical consulting is for.