The statistics that get dental papers rejected

Your manuscript is finished and one doubt will not quite go away: the journal you are aiming at publishes mixed models and split-mouth trials, and you have run a Student's t on 120 teeth. Or worse, the report has already landed and Reviewer 2 has written the sentence everyone in dentistry reads once in their career: teeth are not independent observations. The decision in front of you is not whether the analysis is wrong, because by now you suspect it is. It is how much work the fix takes, whether the fix changes your result, and whether it pays to find that out yourself rather than hear it with a rejection attached.

120 teeth from 30 patients are not 120 data points

Two molars from the same mouth resemble each other more than two molars from two different mouths. They share saliva, hygiene, diet, occlusion, smoking, the same immune system and, almost always, the same operator. That is not a philosophical objection: it is a correlation you can measure, and it has a name, the intraclass correlation. It also has an uncomfortable consequence: every extra tooth you take from a mouth you have already sampled carries less information than a new patient does.

Put numbers on it. You have 30 patients and you measure four teeth each: 120 teeth. If the intraclass correlation between teeth from the same patient is .30, an unremarkable value for periodontal measurements, the design effect is 1 + (m - 1)ρ, that is 1 + 3 × .30 = 1.90. Divide: your 120 teeth carry as much information as 63 independent teeth. And the standard error of your comparison is not the one the software printed, it is that one multiplied by the square root of 1.90, some 38 % larger.

Here is what happens to your p. A comparison the software hands you as t(118) = 2.62, p = .010 becomes, once the unit is counted properly, something in the region of t(28) = 1.90, p = .068. The effect has not changed size or direction: what has changed is how much confidence you were entitled to. That is precisely the paper that falls apart in review, and it falls apart for good reason.

This is not one reviewer's private obsession. The Cochrane Handbook puts it in writing in its chapter on variants of randomized trials: when an intervention is applied to several parts of the same person (it gives both eyes and several teeth as examples) and the analysis is done by unit without accounting for the fact that those units are clustered within the participant, a unit-of-analysis error occurs; and ignoring the clustering makes confidence intervals artificially narrow and p values artificially small (checked in September 2026).

Your degrees of freedom give you away

You do not have to dig into the syntax to find this error: it sits on the surface of the manuscript. Your methods section says you recruited 30 patients. Your results table says df = 118. There is only one story that connects those two numbers, and a reviewer reconstructs it in ten seconds without opening your data and without being a statistician.

The same numbers give you away in three other places. The n under each table, which counts patients in one place and teeth in another within the same paper. The power analysis, computed on 120 when what you recruited or randomized was 30. And the flow diagram, if your trial carries one: if patients go in at the top and sites are analyzed at the bottom, there is a jump somebody has to justify in writing.

It is worth understanding why this weighs so heavily on the editorial decision. A unit-of-analysis error is not fixed by a language edit or an extra paragraph of discussion: it forces a reanalysis, and the reanalysis can change the paper's conclusion. An editor looking at that is not looking at a minor revision, they are looking at a manuscript whose headline result is unsettled. It is one of the reasons a paper that is sound in every other respect gets turned down by a Q1.

Three honest ways out, and which one is yours

Aggregate to the patient. You compute the mean, the proportion or the worst value across each patient's teeth and analyze 30 data points. You lose apparent power, but the power you lose was never real. If your question is about people ("does this treatment improve the patient's periodontal health?"), this is not an emergency workaround: it is the correct answer, and the easiest to defend, because it does not depend on a reviewer trusting your model.

A mixed model with patient as a random effect. You keep the tooth as the unit of observation and you tell the model which mouth each tooth came from. This is what you want when the question is about teeth and you have tooth-level covariates: position, baseline probing depth, restoration type, vitality. Report the patient-level variance or its ICC, because it is information rather than filler: it is the evidence that the clustering lives inside the model and not just in a sentence. If your question is population-level and the individual patient is not of interest, a GEE with an exchangeable working correlation does a comparable job.

Analyze the within-patient difference. This is what you want when the site rather than the person was randomized, which is the split-mouth case. It gets its own section below, because it comes with traps the other two do not share.

Whichever you choose, write it out by name in the methods. "A linear mixed model was fitted with patient as a random intercept" is a two-line sentence that saves an entire review round. If it is not there, nobody will assume you did it: the absence of the sentence reads, quite reasonably, as the absence of the model.

"Two calibrated examiners" is not a result

That sentence appears in hundreds of dental manuscripts and says nothing checkable. Calibrated against what standard? On how many cases? With what result? A reviewer cannot evaluate a claim with no number in it, so they do the only thing available: they ask for it. And the request arrives in the first round, when you can no longer re-examine anybody because fieldwork finished eight months ago.

The scale decides the coefficient, not the habits of the lab. Cohen's kappa for nominal variables: caries yes or no, bleeding present or absent. Weighted kappa for ordinal ones, which is most of yours (ICDAS, plaque indices, mobility grades); unweighted, confusing a grade 1 with a grade 2 counts exactly as much as confusing it with a grade 4, and that is simply untrue. ICC for continuous measures: probing depth in millimeters, clinical attachment level, measurements on images. Bland-Altman when what you are comparing is two measurement methods rather than two people. Raw percentage agreement replaces none of the four, because chance agrees too.

Three traps that cost you a revision round. Kappa depends on prevalence: in a sample where the lesion is rare you can have 92 % agreement and a mediocre kappa, and that does not mean your examiners measure badly. The ICC comes in models (1,1), (2,1) and (3,1), absolute agreement versus consistency, and single versus average measures; publishing "ICC = .87" without saying which is like publishing a mean without units. And the confidence interval is not optional: with 15 repeated cases, an ICC of .87 can carry an interval reaching down to .60, and that lower figure describes your calibration far better than the point estimate does. It is broken down with worked examples in how to report ICC and interrater agreement in APA 7.

Report intra- and inter-examiner reliability, how many cases were repeated, how much time passed between the two readings, and whether the second pass was blind to the first. This is not excessive zeal: it is what the reference guideline for reliability and agreement studies asks for, GRRAS by Kottner and colleagues (Journal of Clinical Epidemiology, 2011), listed on the EQUATOR Network (checked in September 2026). And mind the order: calibration is done and reported before the main study, not bolted on as an appendix computed when a reviewer asks.

Split-mouth: the design is fine, the analysis usually is not

The split-mouth design is one of the best methodological ideas your field has. Each patient serves as their own control, so all the between-person variability (hygiene, diet, saliva, genetics, smoking, adherence) drops out of the comparison. That is why it needs fewer patients than a parallel trial for the same power. That saving is the only reason you chose it.

And it is exactly what gets thrown away when the analysis is wrong. Comparing the test side with the control side using an independent-samples t test, or a Mann-Whitney U, treats the two sides as if they came from two different groups of people. The pairing disappears from the arithmetic and the advantage of the design goes with it: you have kept the cost (fewer patients, more assumptions) and given up the benefit. The unit of analysis in a split-mouth study is the within-patient difference: a paired t test, a Wilcoxon signed-rank test, or a mixed model with patient as a random effect. And what gets randomized is the side, patient by patient, not the patient.

The design also has limits you have to argue for rather than assume. Carry-across is the first: an antiseptic, a systemic drug or a change in brushing habits crosses the midline, and if you cannot defend the claim that it does not, the design is not valid for that intervention. The second is loss to follow-up: in a split-mouth study what you lose is a whole pair, not a site, and dropping only the missing side breaks the pairing of what remains. The third is generalization: you have estimated an effect within mouths, not between patients, and the discussion has to say so in those words.

If the journal asks you for CONSORT, the version you need is not the general one. CONSORT has a specific extension for trials randomized within the same person, the family split-mouth belongs to: "CONSORT 2010 statement: extension checklist for reporting within person randomized trials", by Pandis, Chung, Scherer, Elbourne and Altman, published in the BMJ in 2017 (357:j2835) and listed both on the CONSORT site and on the EQUATOR Network, with its own checklist and its own flow diagram (checked in September 2026). Filling it in before you write the methods saves you reconstructing them afterwards.

The same thing happens in implants, in vitro and imaging

Implants. A Kaplan-Meier curve at implant level, when many patients carry three or four, is the same error in different clothes. Failures within one patient are rarely independent: they share bone, surgeon, loading protocol, smoking and plaque control. What you want is a shared frailty model or a Cox regression with a robust variance clustered by patient, plus a sentence in the methods explaining why. And watch the mirror image of the same problem: median follow-up is computed over patients, not over implants.

In vitro. This is where the error turns invisible, because the word "patient" appears nowhere. Sixty specimens sectioned from fifteen extracted teeth are fifteen clusters, not sixty observations. The same goes for several samples from the same block of material, the same batch of cement or the same thermocycling run. The cluster is whatever the units share with each other; find it before you count the n, not after the ANOVA comes out looking good.

Measurement and imaging. Two problems slip in together here. Two-level nesting (several measurements of the same site, several sites in the same mouth), and the confusion between correlation and agreement: comparing a periapical radiograph with CBCT using a Pearson r can hand you .95 alongside a systematic bias of half a millimeter in every single measurement. Correlation measures whether the two rise together; agreement measures whether they say the same thing. For the second, Bland-Altman, with its mean bias and its limits, and a sentence on whether that bias is clinically tolerable in your context.

What you can check yourself before you submit

One. Copy the n from your methods and every df from your results onto one sheet. If the degrees of freedom talk about teeth and the methods talk about patients, you have located the problem and you needed nobody to find it. Two. Look for the sentence that names the clustering. If it is not written down, it does not exist for whoever reads you. Three. Every reliability coefficient with its model, its interval and the number of repeated cases. Four. If it is split-mouth, that the test is paired and that the randomization of the side is described. Five. Recompute one p value by hand from its own statistic and its own degrees of freedom, and see whether it matches the printed one.

The fifth point is tedious, which is why almost nobody does it to their own manuscript. It is what Previum does with code: it recomputes every p from the degrees of freedom you wrote, checks effect sizes against the F or the t in your own table, and verifies references one by one against eight databases. The prose of the report is written by a model, but around numbers it cannot change. You can read a full example report before uploading anything of your own.

And the limit, said out loud before you discover it yourself: a tool that reads your manuscript will tell you that a p value does not match its degrees of freedom, and that those degrees of freedom do not match the sample you declared, which is exactly the signal this article is about. What it cannot know is whether your model carried a random intercept, because that is not in the numbers: it is in your methods, and writing it is still your job. Nor is it peer review, which is worth stating plainly: it is self-assessment of your own manuscript before you submit. The full routine, tool or no tool, is in checking your statistics before you submit.

Several of these problems never reach a reviewer: they end in a desk rejection, with the editor returning the dental paper in days. Before you submit, run a desk-reject check on your manuscript.

Frequently asked questions

Can I analyze at tooth level if my question is about teeth?

Yes, and often you should. What you cannot do is analyze at tooth level as if teeth were independent. Before you touch the software it helps to separate three units: the unit of the question (are you interested in the patient or the site?), the unit of randomization or sampling (what did you allocate or recruit?) and the unit of analysis (what do the rows of your dataset represent?). When the first is the tooth, you analyze teeth with a model that knows which mouth each one came from. The error is never choosing the tooth: it is forgetting the mouth.

How many patients do I need for a mixed model?

More than people think, and fewer teeth than people think. What a mixed model estimates badly with sparse data is not the fixed effect, it is the patient-level variance, and that variance needs clusters, not more observations inside each cluster. With well under twenty patients the random component comes out unstable and the degrees of freedom become arguable. There are two reasonable ways out: aggregate to patient level and run a simple, honest analysis, or use small-sample corrections (Kenward-Roger, Satterthwaite) and say in the methods which one you used. Adding teeth fixes nothing, because it adds no clusters.

Kappa or ICC?

The scale decides. Nominal, Cohen's kappa. Ordinal, weighted kappa, and state whether the weighting is linear or quadratic because it changes the number. Continuous, ICC with the model spelled out. More than two examiners on a nominal variable, Fleiss' kappa. And if what you are comparing is two devices or two measurement techniques rather than two people, the framework is not a single coefficient but Bland-Altman: mean bias and limits of agreement.

What kappa value does a journal require?

Almost none require one. The Landis and Koch labels, the ones running from "poor" to "almost perfect", come from a 1977 Biometrics paper and are a reading convention, not an editorial requirement, however often reviewers cite them as though they were. What you will be asked for is that the number comes with its confidence interval, that it matches the scale you actually measured, and that it is coherent with the prevalence in your sample. A κ = .62 properly reported over 40 repeated cases is more convincing than a κ = .91 with no interval, no model and no statement of how many cases it was computed on.

Is the split-mouth design always better?

No. It is better when the effect is local and does not cross the midline, when the condition presents bilaterally and comparably, and when follow-up is short. It is worse when the intervention has a systemic effect, when the patient may change behavior across the whole mouth because they know something is being tested on one side, or when you expect substantial loss to follow-up: each loss costs you a whole pair, not half a data point. If you cannot defend the absence of carry-across in the discussion, a parallel trial with more patients costs more and gets published.

I already submitted with teeth as independent. What now?

If there is still time, redo the analysis and tell the editor yourself before anyone asks. If the comment has already arrived, the response that works is the one that redoes the analysis, reports the new result even when it has stopped being significant, and explains in two sentences what changes in the conclusion. A result that shrinks once you do it properly is not a career disaster: it is a different finding, smaller and true, and it can be defended. What does not work is arguing that in your case teeth are nearly independent: the reviewer has the literature on their side and you have the arithmetic against you.

Keep the cheapest check of all, the one that survives any change of software and needs no ability to write a mixed model: take the n from your methods and the degrees of freedom from your main table, and look at them side by side for one minute. If the methods count patients and the table counts teeth, your paper has a problem a reviewer will see before you do. Far better to see it first.