Before You Analyse: Reverse Items, Sum or Mean, and Missing Items

The step that comes before everything else is the one nobody teaches. Before the t test, the ANOVA and the structural equation modelling there are three decisions about the data matrix — flipping the reverse-worded items, choosing whether the score is a sum or a mean, and deciding what happens when someone leaves two boxes blank — that take five minutes, are usually made without thinking, and can quietly ruin everything downstream. An alpha of .17 that should have been .81, a whole participant dropped over one blank box, a coefficient that cannot be compared with the paper you are citing. Here are the three decisions, each with the check that confirms it went right. The wider map of what to verify before you analyse is in how to check statistical assumptions.

How to reverse-score an item (and why it is not maximum − x)

A reverse item points the opposite way to the scale it belongs to. On a self-esteem questionnaire where scoring high means having high self-esteem, “at times I think I am no good at all” is scored high by the people doing worst. They are there on purpose, to break acquiescence — the habit of ticking the same column without reading — and they work perfectly well as long as somebody remembers to flip them before adding anything up. Otherwise each reverse item subtracts where it should add, and the scale measures partly one thing and partly its opposite.

The formula is (theoretical maximum + theoretical minimum) − score. On a 1 to 5 Likert that is 6 − x: 1 becomes 5, 2 becomes 4, 3 stays where it is, 4 becomes 2 and 5 becomes 1. On a 0 to 4 scale it is 4 − x; on 1 to 7 it is 8 − x; on 0 to 3 it is 3 − x. The most repeated mistake is using maximum − x on its own: on the 1 to 5 scale that returns values from 0 to 4, a variable on a different range that no longer lines up with the other items and drags the total down by one point per reversed item.

The second mistake is quieter: using the observed maximum instead of the theoretical one. If nobody in your sample ticked 5 on that item, the column maximum is 4, and an automated recode that reads the data rather than the manual will apply 5 − x to everyone. Nothing warns you, the range looks reasonable, and the scale is mis-scored for all 214 participants. The minimum and maximum come from the questionnaire, never from the matrix.

Always recode into a new variable, with a suffix you can see: se3_R. In SPSS, COMPUTE se3_R = 6 - se3. followed by EXECUTE.; in R, dat$se3_R <- 6 - dat$se3. The reason is not tidiness. If you overwrite the column, three weeks later there is no way to tell whether the job is done, and running the same syntax twice returns the original values — 6 − (6 − x) = x — without one line of warning in the console.

Put the reversed items in the Method, with their numbers: “items 3, 7 and 9 were reverse-scored before the scale score was computed”. It is one line, and it is the difference between a reader being able to reproduce your scale and having to take your word for it.

The check: a negative item-total correlation means you did not reverse it

The corrected item-total correlation is the correlation between an item and the sum of the remaining items; corrected means exactly that, the item does not count itself. On a scale where everything points the same way they all have to come out positive, and the usual convention asks for values above .30 as well. A negative one is, ahead of any other explanation, a reverse item that was never flipped.

With numbers. A 10-item scale answered from 1 to 5, N = 214, with three negatively worded items: 3, 7 and 9. If the average inter-item correlation is .30, the correctly scored version gives α = .81 and corrected item-total correlations of around .48; leave those three unreversed and item 3 comes out at r = −.40 while alpha collapses to α = .17. Not one datum is missing: the information was there all along, it was simply entering with the wrong sign. The extreme case is a negative alpha, mathematically possible when the average inter-item covariance is negative: on a ten-item scale with five negatively worded items, such as the Rosenberg self-esteem scale, that same average correlation leaves the coefficient at α = −.48.

There is an arithmetic check that takes five seconds and needs no software. The mean of the recoded item has to equal (maximum + minimum) minus the mean of the original, and the standard deviation cannot change at all. If item 3 had M = 2.10 and SD = 1.04, the recoded version has M = 3.90 and SD = 1.04. If the SD has moved, whatever you did was not a reverse-score.

To see the whole table — corrected item-total correlation, alpha if item deleted, mean and standard deviation for each item — paste your matrix into the item analysis calculator and you get it on one screen with the signs in plain view. How that coefficient is then written up is covered in how to report Cronbach's alpha in APA 7.

Be careful with automatic reversal. psych::alpha() takes a check.keys = TRUE argument that flips any item correlating negatively with the total and prints a note saying which ones it touched. Treat it as a diagnostic, not as a scoring step: it decides from the data rather than from the wording, so in a small sample, or with an item that is simply bad, it can flip the wrong one and hand you a scale that looks repaired. Read which columns it changed, then do it yourself against the questionnaire manual.

Sum or mean: when it makes no difference and when it does

The mean is the sum divided by the number of items. That is a linear transformation with a positive constant, so almost everything you care about never notices: the correlation with any other variable is identical, the t is identical, the p value is identical, the standardised coefficient is identical and alpha is identical. With the scale from the previous section, M = 3.42 and SD = 0.68 as a mean are M = 34.20 and SD = 6.80 as a sum, and the correlation with the criterion is r = .38 in both metrics.

What does change is the unstandardised coefficient. In a simple regression on that criterion, run on the 211 participants who keep a score once the missing-item rule of the next section is applied, the mean score gives B = 4.75 and the sum score gives B = 0.475, ten times smaller for the ten items, while β = .38, t(209) = 5.94 and p < .001 stay exactly the same either way. That is why comparing your B with the B of another paper using the same instrument means nothing until you know whether they summed and you averaged.

Sum when you are going to compare against something published. Clinical cut-offs and normative tables usually live in the sum metric, and to say what proportion of your sample exceeds a threshold you have to be on the scale where that threshold was defined. Converting between metrics is trivial — multiply by the number of items — but it has to be done: “my mean is 1.2, so my sample is below the PHQ-9 cut-off of 10” is an intuition, not a comparison; 1.2 across the nine PHQ-9 items is 10.8 as a sum, just above it.

Mean when you put subscales of different lengths side by side, or when you want the number itself to keep meaning something. A 4-item subscale and an 8-item one have sums topping out at 20 and 40; on the same figure, the second looks twice as large without being so. As means they both live between 1 and 5 and can be read together. And M = 3.42 on a 1 to 5 scale tells any clinician where the average response of your sample falls; 34.20 tells them nothing until they do the division.

The third difference is the one that really bites, and it is the next section: a sum needs every item present, while the mean of what was answered tolerates gaps. Whichever you pick, declare it in the Method with its range — “the ten items were averaged (range 1–5)” or “the ten items were summed (range 10–50)” — and keep the same metric in the text, the tables and the figures. A text that says 3.42 and a table that says 34.20 for the same variable is a guaranteed request for clarification.

Missing items inside a scale: the 80% rule and prorating

Two different problems share one name. A participant who skips a whole page and leaves the scale blank is a missing value on the variable, handled with the usual machinery — multiple imputation, full information maximum likelihood, and the arguments about the mechanism behind them. A participant who answers eight items out of ten is a much cheaper problem: you have eight of their responses and all you lack is a sensible way to summarise them.

Prorating means averaging what was answered and, if you want the sum metric, multiplying by the total number of items. Someone who answered 8 of 10 with a mean of 2.75 gets a prorated total of 27.5. Underneath sits an assumption worth saying out loud: that the items this person answered stand in for the ones they did not. With one or two gaps that is defensible; with five you are writing half the scale in the participant's name.

The most widely used convention is to prorate when at most 20% of the items are missing and to leave the score missing when more are. On a 10-item scale that is 2 gaps; on a 5-item scale, one; on a 20-item scale, four. It is a convention rather than a law, and the item count matters: on a 4-item subscale a single gap is already 25%, and the prorated score rests on three responses. Some instruments carry their own written rule — the SF-36 and its half-scale rule is the classic example — and that rule outranks any general criterion. Look for it in the manual before inventing one.

The software prorates on its own, and that is the trap. In SPSS, the sum written with the operator (v1 + v2 + ... + v10) returns missing as soon as one item is missing, but the SUM(v1 TO v10) function says nothing about it: it adds up whatever is there, which is exactly treating each gap as a zero. MEAN(v1 TO v10) does the same for the average, even with one response out of ten. The correct form is the suffix, which demands a minimum number of valid values: MEAN.8(v1 TO v10) only computes when at least eight are present, and if you want the sum metric you prorate by multiplying, COMPUTE total = 10 * MEAN.8(v1 TO v10)., never by adding up what was answered, because SUM.8 returns the sum of those eight items rather than the prorated score. In R, rowMeans(x, na.rm = TRUE) holds the same trap and needs a counter in front of it, along the lines of rowSums(!is.na(x)) >= 8, while rowSums(x, na.rm = TRUE) is the zero version: that eight-item participant gets 22 instead of 27.5, five and a half points lower for two blank boxes.

Report it in one sentence: “When a participant answered at least 80% of the items in a scale, the score was computed by averaging the available items (11 participants, 5.1%); below that threshold the score was left missing (3 participants, 1.4%).” If you want the question closed for good, add the sensitivity analysis on the next line: repeated on the 200 cases with complete scales, r = .37 against the r = .38 of the 211 with a valid score. It costs a minute and saves a round of review.

Write the rule before you look at the results. Choosing the threshold once you know which side of .05 your test lands on under each option is not a methodological decision, it is a researcher degree of freedom, and it belongs to the family of choices described in common errors in interpreting statistical results.

Reverse-scoring does not remove the wording effect

With the items flipped and the scale behaving, one phenomenon still surprises people the first time. Negatively worded items tend to keep sharing something with each other that is not the construct: in an exploratory factor analysis a second factor turns up made of exactly the items you recoded. It has been documented for decades, most famously with the Rosenberg self-esteem scale (Marsh, 1996; DiStefano & Motl, 2006).

What that factor is not: evidence that you are measuring two constructs. Before naming it “negative self-esteem” and writing half a discussion around it, look at the list of items loading on it. If it matches the list of items you reversed, the honest reading is a method effect — how the item is worded, not what it measures — and that is how it should be described in the manuscript.

In a confirmatory model it is handled without splitting the construct: let the residuals of the negatively worded items correlate with one another, or add a method factor that only they load on. Fit improves, the substantive structure stays unidimensional, and the decision takes two sentences, with its theoretical justification in front of it rather than behind. How the full factor solution is written up is covered in how to report an exploratory factor analysis in APA 7.

One nuance confuses a lot of people and is worth holding on to. A factor model does not need the recoding at all: an unreversed item simply gets a negative loading and model fit is identical, χ² included, because the model only needs to know how the items covary. The recoding is compulsory for the score — for the sum, the mean, alpha and anything else that aggregates items — and there a flipped sign wrecks the result. Both sentences are true at once: the model does not care, and the sum very much does.

The order of operations, and what reviewers flag

The order is: fix the theoretical minimum and maximum from the questionnaire, reverse the relevant items into new variables, check the item-total correlations and alpha, apply the missing-item rule, compute the score, and only then analyse. Everything downstream of a mistake in step two is contaminated, and no statistical test will catch it for you: a t computed on a mis-scored scale comes out just as pretty, p value and all.

Always work from the raw file. A script that recodes a column in place and gets run twice leaves the data exactly as they started, and the symptom is disorienting: the alpha that was α = .81 yesterday is α = .17 today on the same data with the same syntax. If your workflow is “open the .sav and start clicking”, that failure is only a matter of time.

The first thing a reviewer looks at is a low alpha on a well-established scale. If the instrument has been publishing coefficients above .85 for twenty years and yours lands at .17, the likeliest hypothesis is not that your sample is unusual, it is that some items were never reversed. The second is a Method section that does not say which items were reversed. The third is a metric that does not add up: M = 3.42 in the text and M = 34.20 in the table for the same variable, or a range that is impossible for the number of items you declare.

The fourth is silence about missing items. If N changes between two tables with no sentence explaining it, you will be asked; and if it never changes at all even though the questionnaire ran to thirty items and was collected on paper, you will be asked too. Neither is an accusation of anything: three lines of Method are missing.

Three checks, five minutes. The mirrored means of the recoded items, a column of item-total correlations with no negative sign in it, and the count of participants affected by the missing-item rule. Done before the first test they are done once; done after the Results are written, the paper gets rebuilt.

Frequently asked questions

How do you reverse-score an item?

With the formula (theoretical maximum + theoretical minimum) − score. On a 1 to 5 Likert that is 6 − x, on a 0 to 4 scale it is 4 − x, and on a 1 to 7 scale it is 8 − x. The maximum and minimum come from the questionnaire, not from your data: if nobody ticked 5 in your sample and you use the observed maximum, you mis-score everyone. Save the result in a new variable with a suffix such as se3_R, because running the recode twice on the same column silently returns the original values.

How do I know if I reverse-scored an item correctly?

Look at the corrected item-total correlation: if it is negative, that item is still unreversed. On a 10-item scale answered by 214 people, with three negatively worded items left unrecoded, item 3 can give r = −.40 and hold alpha at .17; with the three recoded, that item goes to r = .48 and alpha rises to .81 with no data changed. The quick check is arithmetic: the mean of the recoded item must equal (maximum + minimum) minus the original mean, and the standard deviation cannot change. If the original had M = 2.10 and SD = 1.04, the recoded item has M = 3.90 and SD = 1.04.

Should I sum or average the items in a scale?

For most analyses it makes no difference, because the mean is the sum divided by the number of items: the correlation, the t, the p value, the standardised coefficient and alpha are identical either way. It matters in three places. Sum if you will compare against published cut-offs or norms, which usually live in that metric. Average if you place subscales of different lengths side by side, or if you want the number to be read on the response scale. And watch the unstandardised coefficient: with ten items, B = 4.75 on the mean is B = 0.475 on the sum.

How many missing items can a scale tolerate?

The most widely used convention is to prorate when at most 20% of the items are missing and to leave the score missing when more are: on a 10-item scale, up to 2 gaps. Prorating means averaging the answered items and, if you want the sum, multiplying by the total number of items. Be careful with your software's sum function: adding up only the answered items is the same as treating every gap as a zero. If the instrument carries its own rule in its manual, that rule outranks any general convention.

Why is my Cronbach's alpha negative?

Almost always because reverse-worded items were never recoded. Alpha can come out negative when the average inter-item covariance is negative, and the commonest way to produce that is to add items pointing the opposite way into the same total: on a ten-item scale with five negatively worded items, leaving them all unrecoded puts the coefficient at around −.48. Before concluding that the scale does not work, read the column of corrected item-total correlations and look for negative signs.

Do you reverse-score items before factor analysis?

The model does not need it: an unreversed item simply gets a negative loading and the fit is identical, because factor analysis works from the covariances. The score does need it, because sums, means and alpha aggregate the items and one flipped sign ruins them. Reverse-scoring also does not remove the wording effect: an exploratory analysis often shows a second factor made of exactly the reversed items, which is a method effect rather than two distinct constructs.

With the reverse items recoded and checked, the metric declared and the missing-item rule written down before you saw the results, the step before analysis is done and everything after it rests on something solid. When the measurement gets harder — subscales of different lengths, short forms, repeated measures where the gaps pile up wave after wave — we can work through it together in my statistical consulting.