Questionnaire and scale validation for health research: content validity (Aiken's V, CVI), reliability (omega, alpha), CFA and invariance, reported to COSMIN.
From expert-panel content validity (Aiken’s V, CVI) to reliability, factor analysis, measurement invariance and convergent and discriminant validity, reported in line with COSMIN: with the rigour peer reviewers expect and in the right software.
The process journals expect in psychometrics and health research, from content validity to criterion validity, mapped to the COSMIN measurement properties.
The rating form for your experts, Aiken’s V with its confidence interval and the content validity indices (I-CVI and S-CVI).
The structure of the instrument and the fit of the theoretical model with CFI, TLI, RMSEA and SRMR.
Internal consistency with omega, which does not assume equal loadings, and the alpha many committees still ask for; test-retest stability with the ICC when there are two measurements.
Evidence that the instrument works the same across groups (sex, age, country or language) before you compare them.
Correlations with external scales, AVE and HTMT as construct-validity evidence.
How the instrument relates to relevant criterion variables: regression, ROC curves and cut-off points.
Tables, syntax and an explanation of every index. For a paper or a doctoral thesis, the section drafted for you to review and make your own; for a dissertation that is assessed as your own work, you write it and I review your draft.
Content validity is the first thing a validation is asked for: a panel of experts confirming that each item measures what it should. It is collected with a rating form and summarised with Aiken’s V and the content validity indices.
Lynn (1986) recommends at least three experts, and more than ten rarely adds anything. Most theses use between five and ten, chosen for their training and their experience with the construct.
If your study uses a new or adapted questionnaire, validating it is a well-defined job: the scope is agreed up front and you get a fixed quote in writing before any work starts.
Choosing the wrong technique is the most common mistake in questionnaire validation.
An exploratory data-reduction technique: it summarises variables into components, but it does not test a theoretical model or give fit indices.
The current standard in psychometric validation. It provides CFI, RMSEA and SRMR and allows measurement invariance across groups.
Tell me about your instrument and I will walk you through the complete validation process.
Exploratory factor analysis (EFA) is used to discover the underlying structure of a scale when you have no clear prior hypothesis. Confirmatory factor analysis (CFA) tests whether a structure defined in advance, based on theory or a previous study, fits the data. For validating an instrument with an expected structure, CFA is the appropriate standard; if you are developing a scale from scratch, you usually start with EFA and confirm with CFA in an independent sample.
Lynn (1986) recommends at least three experts, and more than ten rarely adds anything; most theses use between five and ten. The most requested index is Aiken's V per item with its confidence interval, alongside the item-level CVI (I-CVI) and the scale-level S-CVI/Ave. The rating form collects the relevance, clarity and fit of each item.
It depends on the number of items and factors and the complexity of the model, not on a single fixed rule. As a guide, several hundred participants are usually recommended for a robust CFA, but the exact figure is set from your design. In the free consultation I assess your case and tell you the realistic sample you need.
McDonald's omega and Cronbach's alpha for reliability, CFA fit indices (CFI, TLI, RMSEA, SRMR), measurement invariance across groups when relevant, and convergent, discriminant and criterion validity evidence. Everything is organised around the COSMIN measurement properties and reported to the standard Q1/Q2 journals expect.
Yes. Measurement invariance (configural, metric and scalar) is essential before comparing groups or, in cross-cultural adaptations, before comparing scores across languages or countries. I run it and interpret it so your comparisons are valid.
Mainly R (lavaan, semTools, psych) and Mplus for CFA, SEM and invariance, and JASP or jamovi when a more visual workflow is preferred. The choice adapts to your design and to what your supervisor or target journal expects.
For a paper or a doctoral thesis I deliver the section drafted in APA 7 for you to review with your co-authors or supervisor and make your own. For a master’s dissertation or any work assessed as your own, I run and explain the analysis and give you the tables and syntax; you write it and I review your draft.
Yes. I support the statistical side of adapting and validating an instrument in a new language or culture: structure, reliability, measurement invariance against the original version, and the reporting required for publication.