How to Report Aiken’s V for Content Validity in APA 7

You walk into the viva with a table where every V clears .80, and someone on the panel asks how many judges you used. Five. That is where the defense of your instrument ends, because with five judges the interval around that V is more than thirty hundredths wide — a V = .80 from five judges on a 1–5 scale carries a 95% CI of [.58, .92] — and the table stops proving what it claimed to prove. Aiken’s V is a good index and the default choice for expert review in psychology and the health sciences, but it is almost always reported badly: no interval, no mention of how many categories the rating scale had, and .80 quoted as though APA had written it into the manual. Here is the formula, the panel size that survives a question, the interval hardly anyone includes and the exact sentence for your Method section. The wider map of expert-review methods is in questionnaire validation by an expert panel.

What Aiken’s V measures, and what it does not

Aiken (1980, 1985) proposed V to answer one specific question: given an item and a group of judges rating it on an ordinal scale, what proportion of the maximum possible endorsement did that item obtain? The judges do not answer the questionnaire, they judge it, item by item, usually on several criteria at once — relevance, coherence with its dimension, clarity of wording and sufficiency of the set. Each criterion yields its own V, and collapsing them into a single figure is the first way people throw information away.

The formula is V = S / [n(c − 1)], where S is the sum of the ratings once the lowest scale point has been subtracted, n the number of judges and c the number of categories. Take eight judges on a 1–5 scale rating one item 5, 5, 5, 5, 4, 4, 4 and 4. Subtracting the minimum leaves 4, 4, 4, 4, 3, 3, 3, 3, so S = 28. The maximum possible is n(c − 1) = 8 × 4 = 32. Hence V = 28/32 = .88.

The most common arithmetic slip sits in c. That number belongs to the scale the judges used, not to the questionnaire you are building. If your participants will answer on a 5-point Likert but the judges rated each item from 1 to 4, then c is 4 and the denominator is n × 3. Putting 5 there inflates the denominator and drags every V down at once, and because the distortion is constant nobody spots it by looking at the table.

There is also something V does not measure: agreement between judges. It is a rescaled mean, and identical means hide very different pictures. With six judges on a 1–4 scale, an item rated 4, 4, 4, 3, 3, 3 gives V = .83; an item rated 4, 4, 4, 4, 4, 1 gives exactly the same V = .83. In the second one an expert thinks the item is irrelevant, and that fact vanishes into the coefficient. Always report the standard deviation or the range of the ratings alongside V.

How many judges you actually need

The standard reference is Lynn (1986): a minimum of five experts and a recommended range of five to ten. Below five the calculation does not break, it simply stops discriminating; above ten the returns flatten and coordinating the panel becomes a logistical problem in its own right. Five is the floor, not the target.

What really changes with panel size is precision, and it shows most clearly when the coefficient is held constant. An item with V = .90 on a 1–5 scale carries a 95% CI of [.70, .97] with five judges, [.77, .96] with ten and [.81, .95] with twenty. The point estimate is identical in all three; what differs is that only with twenty judges can you claim the item clears .80. With five, the same figure is compatible with a mediocre item.

Two points about who sits on the panel. First, you do not need an odd number. That rule comes from majority voting and has nothing to do with V, which breaks no ties. Second, mix content experts (clinicians, researchers in the area) with measurement experts (psychometricians, methodologists). A panel of five clinicians will catch clinically irrelevant items and wave through double-barrelled ones, ambiguous wording and impossible response options.

Judges work independently and without seeing one another’s ratings, and they need the operational definition of the construct and its dimensions in front of them. Without that specification table they are rating the phrasing of a sentence, which is a different exercise. In the manuscript you describe who they were: how many, their qualifications, their field and their years of experience. “Seven expert judges” and nothing more is the sentence that invites the examiner’s next question.

Where the .80 cut-off comes from

Aiken (1985) did not propose a fixed threshold: he published tables of critical values derived under the null hypothesis that judges rate at random, with equal probability across categories. That critical value depends on how many judges there are and how many categories the scale has. With five judges on a five-point scale, the 5% cut lands exactly on .80 (p = .040), while V = .75 sits at p = .079. That detail explains much of why the number fossilized: early panels tended to have five judges.

Change the panel size and the cut moves. With eight judges on the same five-point scale, V = .75 already falls below 5% (p = .030) and V = .72 lands just outside it (p = .052). Applying .80 to every panel regardless of size is therefore arbitrary: you are over-demanding from a large panel and under-demanding from a small one.

In current practice the defensible framing is a two-step one — ≥ .70 acceptable, ≥ .80 adequate — presented as a convention of the field and not as an APA rule, because APA sets no threshold for anything. In the Spanish-language literature the .80 is normally traced to Escurra (1988), who worked the expert-judge criterion through a binomial approach. If you write “items were retained at V ≥ .80”, say where the number came from.

The threshold should track what the instrument is for. An exploratory scale for a new construct can live with .70 and say so in the limitations. A screening tool that will label somebody as a case in a clinic cannot. And apply the criterion criterion by criterion: an item with relevance at .92 and clarity at .68 has not passed, it is badly worded.

The Penfield and Giacobbi confidence interval

Penfield and Giacobbi (2004) pointed out the obvious thing nobody had done: V is a point estimate obtained from a handful of judges, and without an interval it settles nothing. Because it is also a proportion bounded between 0 and 1, the ordinary symmetric interval will not do — it runs outside the range whenever V is high — so they applied Wilson’s score method, which produces an asymmetric interval that always stays inside [0, 1].

The lower bound is L = [2mV + z² − z√(4mV(1 − V) + z²)] / [2(m + z²)], with m = n(c − 1) and z = 1.96 for 95%; flipping the middle sign gives the upper bound. Worked through: six judges, a 1–4 scale, S = 15, so m = 18 and V = .83. The interval comes out at 95% CI [.61, .94]. The point estimate clears the threshold comfortably and the lower bound sits twenty hundredths beneath it.

Hence the decision rule they proposed, which remains the most defensible one available: judge the item on the lower bound, not on the point estimate. An item is retained without argument when the lower end of its interval clears whatever criterion you set. If the point clears it and the interval does not, the item has not passed — it is undecided, and what is missing is judges.

One clarification that prevents a frequent mistake: this interval is per item. The V for the whole instrument is the mean of its item-level Vs, and this formula does not apply to it, because it is built on the n(c − 1) rating units of a single item. Report the overall figure as a mean with its range and standard deviation. For the item-level computation with the interval included, use the Aiken’s V calculator.

Aiken’s V and Cronbach’s alpha measure different things

This confusion shows up in most theses and it dissolves as soon as you ask which matrix each number came from. V is computed on an items-by-judges matrix: eight experts rating thirty items. Alpha is computed on a participants-by-items matrix: three hundred people answering those thirty items. Two different tables, two different samples, collected at two different points in the study.

They also answer different questions. V is content validity: do these items represent the construct and its dimensions? Alpha is reliability: do responses to these items behave consistently in my sample? And neither rescues the other. A set of irrelevant items can return α = .93 quite happily; it just means they are consistently irrelevant. The reporting rules for the coefficient are in how to report Cronbach’s alpha in APA 7.

A third index gets confused with V and is not it either: inter-rater agreement. If the claim you want to make is that your experts converged, V does not support it — you have already seen two very different patterns produce the same figure. That is what the ICC, kappa and Krippendorff’s alpha are for, since they actually model concordance; the ICC, with its three models and two definitions, is covered in ICC and inter-rater reliability.

There is also a close relative worth keeping separate: if your judges answered yes or no to “is this item relevant?”, the formula does not break, but with c = 2 it collapses to the proportion of judges who said yes, which is exactly the I-CVI (Polit & Beck, 2006); and if the question was whether the item is essential, the index you want is Lawshe’s CVR (1975). Calling a dichotomous rating “Aiken’s V” is not an arithmetic error, it is a label promising an ordinal scale that is not there: keep V for scales of three categories or more and call the rest by their own names, because a psychometrically minded reviewer catches this on first reading.

How to report it, with the sentence and the table

The paragraph belongs in the Method section, in the expert-review procedure, and reads like this: “Eight expert judges (three doctoral-level clinical psychologists, three methodologists and two psychometricians, averaging 12 years of experience) independently rated the 30 items for relevance, coherence and clarity on a 1–5 scale. Item-level Aiken’s V for relevance ranged from .63 to 1.00, with a mean of V = .91. Twenty-six of the 30 items reached V ≥ .80 on all three criteria. Four fell below (V = .63, .69, .72 and .78): two were rewritten on the basis of the judges’ written comments and submitted to a second round, and two were dropped.”

The table that accompanies that paragraph carries one row per item and these columns. Item. The number, plus the wording itself for short instruments. M and SD of the ratings. To one decimal, and they are what reveals a dissenting judge. V by criterion. One column for relevance, one for coherence, one for clarity. 95% CI. In square brackets, two decimals. Decision. Retained, rewritten or dropped.

APA formatting for the coefficient: V is a Latin letter, so it is italicized, unlike alpha and omega, which are Greek and set in roman. Two decimals and no leading zero, because it cannot exceed 1: V = .88, never 0.88. The only exception is perfect endorsement, written 1.00. And it is not a percentage: V = .88 does not mean “88% of the judges”.

What reviewers flag is silence about the items that fell. If you report 26 out of 30, somebody will ask about the other four. Say how many were rewritten, how many were dropped and whether a second round produced new coefficients. And do not close the validation there: V says the experts see the items where you put them, not that the data cluster that way. That is what the factor structure tests, starting with how to report an EFA in APA 7. To get every V with its interval plus the I-CVI and S-CVI indices in one pass, paste the judges’ matrix into the content validity calculator.

Already drafted it? Paste your Results section into Review one section: it reads it the way a reviewer would and returns the first objection, attached to the sentence that causes it.

Frequently asked questions

What is Aiken’s V and what is it used for?

It is a content validity coefficient that quantifies how strongly a panel of expert judges endorses each item of an instrument. It is computed as V = S / [n(c − 1)], where S is the sum of the ratings minus the lowest scale point, n the number of judges and c the number of rating categories. It runs from 0 to 1 and is used to decide which items are retained, which are rewritten and which are dropped before you collect any data.

How many expert judges do you need for Aiken’s V?

Five is the accepted minimum and five to ten the recommended range (Lynn, 1986). You do not need an odd number: that rule comes from majority voting and does not apply here. What changes with panel size is precision, not the coefficient: on a 1–5 scale, V = .90 carries a 95% CI of [.70, .97] with five judges and [.81, .95] with twenty. Same point estimate, different conclusion.

What is an acceptable Aiken’s V value?

The widely used convention is .70 as acceptable and .80 as adequate, and it should be presented as a convention because APA sets no threshold. The .80 comes from Aiken’s (1985) critical value tables, which in fact depend on the number of judges and categories: with five judges on a five-point scale the 5% cut lands on .80, but with eight judges it drops to .75. The most defensible rule is to require the lower bound of the confidence interval to clear your criterion, not just the point estimate.

Is Aiken’s V italicized, and does it take a leading zero?

It is italicized, because it is a Latin letter, unlike alpha and omega, which are Greek and set in roman type. It takes no leading zero, because the coefficient cannot exceed 1: write V = .88, not 0.88. Report it to two decimals, with perfect endorsement written as 1.00. It is not a percentage either: V = .88 does not mean that 88% of the judges approved the item.

What is the difference between Aiken’s V and Cronbach’s alpha?

V is content validity, computed from the ratings a panel of experts gave to the items; alpha is reliability, computed from how participants answered those items. Two different matrices, two different samples and two different stages of the study. Neither substitutes for the other: a set of irrelevant items can return a very high alpha, which only means they are consistently irrelevant.

How do you compute a confidence interval for Aiken’s V?

With the score method of Penfield and Giacobbi (2004), which adapts Wilson’s interval to a coefficient bounded between 0 and 1 and is therefore asymmetric. With six judges on a 1–4 scale it is reported like this: V = .83, 95% CI [.61, .94]. The interval is computed per item; the instrument-level V is the mean of the item-level values and is reported with its range and standard deviation, not with this formula.

With the formula set up correctly, five to ten judges, the Penfield and Giacobbi interval beside every V and the threshold presented as a convention, expert review stops being the weak chapter of the thesis. If your panel has left items in no man’s land, or you are unsure how to fit a second round into the manuscript, that is what my statistical consulting is for.