Writing the statistical analysis plan for a PhD funding application

Every doctoral funding scheme asks for the same thing in different words, and it is not your literature review: it is whether the study you describe can actually be done with the people, the time and the money you are asking for. That question is answered in two or three paragraphs of method, and those paragraphs are usually written last, at midnight, by someone who has spent three weeks polishing the rationale. This article is about those paragraphs. If what you need is the long version — the analysis plan you will write with your supervisor once the money is in — it is in the thesis data analysis plan.

What the panel is actually checking

The person reading your proposal is an academic from your broad field, rarely from your exact topic, with a stack of applications and a few minutes for each one. They are not going to argue with your hypothesis — your supervisor's name on the form covers that. They check three things that show at a glance: that the design can answer the question you wrote, that the sample you promise is reachable inside the funded period, and that the analyses you name match that design and that sample.

Incoherence between those three is what costs you, and it almost always shows up in the same place: the verb in your aim. "To determine the effect of rumination on depressive symptoms" is a causal claim, and a cross-sectional design will not carry it. Either the verb changes — "to examine the association between" — or the design does. It takes ten seconds to fix and it is the first thing visible on a fast read.

How much room you get depends on the scheme. A UKRI or ESRC studentship proposal is usually a couple of pages; an NIH F31 gives you a six-page Research Strategy in which the research training plan is judged on precisely this; an Australian RTP or an ARC-linked scholarship sits somewhere in between. Whatever the limit, budget paragraphs rather than pages for the analysis, and accept the consequence: you either write three dense sentences that can be checked, or you write generalities. "Data will be analyzed using appropriate statistical techniques in SPSS" does not lose points for being dull. It loses points because there is nothing inside it to evaluate.

One more constraint that British and Australian applicants underestimate: the clock. A three-year doctorate that needs ethics approval, twelve months of recruitment, an eighteen-month follow-up and a thesis does not fit, and a panel that supervises students for a living can see it does not fit. Say what happens to the follow-up if recruitment runs late, in one sentence, before somebody asks.

The sample size paragraph almost nobody writes

There are two ways to fail here and they are equally common. The first is justifying nothing: "a sample of approximately 300 participants will be recruited." The second is justifying with no origin: "the sample size was calculated in G*Power, yielding N = 128." Both leave the same hole, and it is exactly the one a reviewer goes looking for: where did the effect size in that calculation come from?

A defensible paragraph has five parts and fits in three lines. The expected effect and its source: a meta-analysis in your area, a comparable published study or, when there is nothing, the smallest effect that would be of interest. The primary contrast: one, the one the thesis rests on, not the fifteen secondary analyses. Alpha, power, and one- or two-tailed. The software and its version, because that is what lets somebody reproduce the number. And the attrition you expect, with the percentage you are inflating by and where it comes from.

Written out, it looks like this: "The sample size was determined for the primary contrast, the between-groups comparison at post-test. Taking the pooled effect from the meta-analysis by [citation] as the reference (d = 0.50), with α = .05 two-tailed and power = .80, G*Power 3.1.9.7 returns 64 participants per arm (N = 128). Allowing for 20% attrition at six-month follow-up, 160 participants will be recruited." Four sentences, all checkable, and anyone with G*Power open recovers the 64 in a minute. The arithmetic behind them is worth seeing once, because it changes what you are willing to promise: that same between-groups test needs 128 participants for d = 0.50 and 352 for d = 0.30, and a multiple regression with five predictors reaches .80 with 92 participants for a medium effect (f² = 0.15) but demands 647 for a small one (f² = 0.02).

The expensive mistake lives in the first part. It is tempting to plug in the effect your pilot produced, but a pilot of 20 people estimates an effect with an enormous confidence interval, and if it came out large that is partly because it came out large: planning on that figure leaves the real study underpowered. When there is no literature, the honest move is to state the smallest effect of interest in those words. If what you have is an effect in a different metric — an r, a partial eta squared, a mean difference with its SD — the effect size calculator converts it into the one your calculation needs, and the long version is in how to calculate the sample size for a thesis.

When the sample is already fixed: sensitivity, not post hoc power

Often you do not get to choose N. The group already has a cohort, or you will be working with whichever patients pass through one clinic, and that is however many it is. Writing that down beats disguising it, and the calculation simply runs backwards: instead of asking how many people you need to detect an effect, you ask which effect you can detect with the people you will have. That is a sensitivity analysis, and it comes out of the same G*Power menu with one option changed.

The paragraph that persuades reads like this: "The available cohort provides 120 participants. With α = .05 two-tailed and power = .80, the study can detect correlations from r = .25 upwards; smaller associations fall outside the reach of this design and will be interpreted through confidence intervals rather than hypothesis tests." Look at what that does. It concedes a limit and proves you calculated it, which is the opposite of hiding it. Panels fund modest samples all the time; what they will not fund is an applicant who appears not to have noticed.

What you never write is post hoc power computed from the effect you observed. It is the p value in a costume — a small p always yields high power — and it carries no new information; a fair share of reviewers know this, and with them it costs you credibility. The reasoning, with the arithmetic in front of you, is in post hoc power analysis.

Do not promise analyses you cannot run

The temptation is obvious: a list of sophisticated techniques looks like ambition. It works the other way round. Every technique you name is a promise the panel can check against your N, against your calendar and against your supervisor's track record, and when one of the three does not hold, what comes across is not ambition but an applicant who does not know what things cost. The rule is short: every analysis you name needs a sample that supports it, a plan B written in one line ("if the model fails to converge, a version with collapsed factors will be estimated"), and somebody who has done it before. That last one is not bureaucracy — your supervisor's CV goes in the same application, and an analysis that appears nowhere in it is an inconsistency between two documents that get read one after the other. These are the four places it breaks.

The SEM that does not fit. A structural model with six latent factors and twenty-five indicators estimated on N = 90 will very likely fail to converge, and if it does converge you will not trust the standard errors. The conventions people cite — ten to twenty cases per estimated parameter, or roughly 200 participants as a floor — are conventions and should be presented as such, but they are enough to see from a distance that the model weighs more than the sample. If the sample cannot grow, write the alternative: path analysis on observed composites, or the measurement model first and the structure afterwards.

The multilevel model with eight schools. Data nested in schools or hospitals is multilevel by design, but the variance at the upper level is estimated from the number of clusters, not the number of participants: with eight schools, eight hundred pupils will not save you. The most cited convention asks for around 30 clusters, and closer to 50 if cross-level interactions matter to you. Below that, the honest route is cluster-robust standard errors or the school as a fixed effect — and saying so in the proposal rather than in the viva.

The two-wave study sold as something else. Two measurements let you estimate a cross-lagged panel, but not separate the stable part of a person from change within that person: that needs three waves at minimum. If the calendar only stretches to two, write two and write what two can conclude. A modest design with its limits drawn is defensible; a modest design sold as a strong one is not.

The "and machine learning" line. It usually appears at the end, once the proposal is written, and almost always with no held-out validation sample, no named person to program it and no statement of which question it answers that the regression two lines above does not. If you cannot answer those three in one sentence, that line subtracts rather than adds.

The chain from aim to design to measure to analysis

A methods section that reads well is a chain with every link visible: aim, the design that permits it, the variable with its instrument, the analysis that produces the answer. Written out it takes very little room. "Aim 2: to examine whether emotion regulation mediates the association between problematic social media use and depressive symptoms in adolescents. Cross-sectional design, N = 400 across twelve state secondary schools. Predictor, mediator and outcome are measured with [instruments, with their published reliability]. A simple mediation model will be estimated with 5,000 bootstrap resamples and bias-corrected percentile intervals."

That chain saves the reviewer work, but mostly it saves you: writing it out exposes your own gaps. An aim with no instrument. An instrument that measures something adjacent to what the aim claims. An analysis you listed out of habit that answers none of the questions above it. If you are unsure which test belongs to which design, the guide is in how to choose a statistical test.

One warning aimed at that example, because it is the most requested design in psychology and health research: the power of the indirect effect is not the power of the total effect. Fritz and MacKinnon (2007) showed that when both paths are small, detecting mediation takes several hundred participants — above 500 — while two medium paths need fewer than a hundred. If your project lives on a mediation, the sample size is calculated for the mediation, not for the correlation you started from.

The last month, in the right order

The order of work is not the one that feels natural. First, close the aims and check their verbs, and decide which contrast is primary, because there is one and everything else is secondary. Second, run the power or sensitivity analysis, write the paragraph and save the G*Power screenshot in the project folder. You will need it again in the thesis and in the first paper, and in two years you will not remember which parameters you entered.

Third, write the whole methods section and put every named analysis through the same question — does my N support it? — deleting the ones that fail. Fourth, read it as if you were the reviewer, and leave room for the administration: your research office almost always sets an internal deadline days before the funder's, and the sign-offs it needs are not in your control.

The check that catches the most costs five minutes: read only the methods paragraphs, out loud, with the list of aims beside you. If there is an aim with no analysis, or an analysis that answers no aim, that is the crack. It is exactly the exercise the panel will run, with one difference — they will do it once and you can do it ten times.

Frequently asked questions

Do I need a power analysis in a PhD funding application?

No scheme has a box labeled "power analysis", but it is the standard way to justify a sample size and it takes two lines. Without it, "we will recruit 300 participants" is a number with no origin that a reviewer cannot check. With it, they check it in a minute and move on, which is what you want. Several funders, NIH among them, explicitly ask reviewers to judge scientific rigor, and the justification of the sample sits inside that judgment.

What effect size should I use if there is no previous research on my topic?

The smallest effect of interest, stated in those words: the smallest difference or association that would change something in practice or in theory. It is a defensible, transparent decision. Cohen's (1988) conventional benchmarks work as a last resort, presented as a convention rather than a law. What you should not use is the effect observed in your own pilot, because small samples inflate it.

How do I justify a sample size when the dataset already exists?

With a sensitivity analysis instead of an a priori one. You state the N you will have, fix alpha and power, and report the smallest effect the design can detect. For example, with N = 120, α = .05 two-tailed and power = .80, the study detects correlations from r = .25 upwards. Then say what you will do with effects below that line: estimate them and report confidence intervals rather than test them.

Can I change the analysis after the funding is awarded?

Yes. A proposal is not a contract and doctoral projects evolve; what you cannot do is invent the justification afterwards. Write the plan B into the proposal itself ("if multivariate normality does not hold, a robust estimator will be used") and preregister the final version on a platform such as OSF once the data collection design is fixed. The change then stops looking suspicious and starts looking documented.

How long should the methods section of a studentship proposal be?

As long as the scheme's word limit allows, which is worth checking before you write rather than after. In practice that means two to four dense paragraphs, and the priority order is design, sample with its justification, primary analysis, and one line of plan B. If something has to go for lack of space, let it be the list of secondary analyses.

Is it a weakness to propose a pilot study first?

Only if the pilot has no decision rule attached. A pilot with stated progression criteria — recruitment rate, retention, whether the measure works in this population — reads as competence. A pilot with no criteria reads as a year of delay. State explicitly that the pilot will not be used to estimate the effect size for the main study, because that is the use reviewers are worried about.

With the design stated, a sample size anyone can recompute and a list of analyses your N can carry, the methods section stops being the weak point of the application and starts being the part that holds it up. If the project gets complicated — nested data, repeated measures, a model you are not sure will converge on the sample you will actually have — we can work through it in a statistical consulting session and you leave with the paragraph written.