Measurement invariance is a concept that should be mandatory for any researcher who compares groups (men and women, different countries, experimental and control groups) using psychological questionnaires or scales. The idea is simple: before comparing the scores of two groups on a scale, you need to ensure that the scale is measuring the same thing in both groups. If the items of a depression questionnaire function differently in men than in women (for example, if the item about crying has a different meaning or factor loading depending on gender), the differences you find in total scores could be due to real differences in depression or simply to the fact that the instrument does not measure the same thing in both groups.
Despite its importance, measurement invariance is evaluated in a very low percentage of studies that make comparisons between groups. Most researchers directly compare total scores or scale means without considering whether the instrument functions equally in the groups they are comparing. This is a serious problem, because violations of invariance are more frequent than commonly thought, especially in cross-cultural comparisons and in scales that assess constructs with social or cultural load.
If you would rather not write the lavaan code by hand, the measurement invariance checker runs the four models (configural, metric, scalar and strict) and gives you the ΔCFI and ΔRMSEA at each step.
Levels of invariance
The evaluation of measurement invariance is carried out through a sequence of multigroup CFA models, each with progressively more stringent constraints. The first level is configural invariance, which simply verifies that the same factor structure (the same number of factors and the same pattern of loadings) holds across all groups. This is the minimum necessary level: if even the basic structure is not the same in both groups, comparing scores is meaningless.
The second level is metric (or weak) invariance, which adds the constraint that factor loadings be equal across groups. This means that a one-unit change in the latent factor produces the same change in the observed indicator in both groups. Metric invariance is necessary to be able to compare relationships between variables (correlations, regression coefficients) across groups. If it holds, you can say that the structural relationships between your variables are comparable, although you still cannot directly compare means.
The third level is scalar (or strong) invariance, which additionally constrains the item intercepts to be equal across groups. This is the level necessary to be able to compare latent factor means between groups. Without scalar invariance, the observed differences in means could be due to differences in item intercepts (what is known as item bias or differential item functioning) and not to real differences in the latent construct. Many researchers are unaware of this requirement and compare means between groups without having verified scalar invariance, which can produce erroneous conclusions.
How to evaluate invariance in practice
The evaluation of invariance is carried out by comparing the fit of successive models. If moving from the configural to the metric model does not produce a substantial deterioration in fit, metric invariance is accepted. The same applies for moving from metric to scalar. The question is how to define "substantial deterioration." The chi-square difference test is the classic criterion, but it is too sensitive with large samples and tends to reject invariance for trivial differences. The most commonly used criteria today are based on changes in practical fit indices: Chen (2007) proposed that a change in CFI of 0.010 or less, combined with a change in RMSEA of 0.015 or less, indicates that invariance holds at that level.
In practice, it is quite common for complete invariance (all parameters equal) not to hold, but for partial invariance to hold. Partial invariance implies that most parameters are equal across groups, but some items are allowed to differ. Byrne, Shavelson, and Muthen (1989) argued that partial invariance is sufficient for making valid comparisons, as long as at least two indicators per factor maintain invariance. However, this minimum criterion has been debated, and the current recommendation is to release the fewest possible constraints and only when there is a clear theoretical or empirical justification for doing so.
Implementation and recommendations
In R, the lavaan package allows you to carry out the entire invariance evaluation process using the measurementInvariance() function from the semTools package, or manually by specifying models with progressive constraints. Mplus is another excellent option with specific syntax for multigroup models. In SPSS, AMOS allows you to perform multigroup CFA, although the interface is less intuitive than lavaan for this purpose.
Before you submit: if your paper compares groups, sooner or later a reviewer will ask whether the scale measures the same thing in all of them. Run the manuscript through the Q1 Paper Reviewer first. It is a free Reviewer 2 style pre-review that catches exactly this type of objection (means compared without having tested scalar invariance, a partial model with constraints released and no justification, a ΔCFI you report but never interpret) before an editor raises it and costs you three months.
My recommendation is clear: if you are going to compare groups on any psychological measure, evaluate measurement invariance before making any substantive comparison. Not doing so is like comparing temperatures measured with thermometers that might be calibrated differently. If invariance holds, you can proceed with your comparisons with the confidence that the differences reflect real differences in the construct. If it does not hold, at least you know that your comparisons have an important limitation that you should acknowledge and discuss in your article. This transparency, although it may seem to weaken your conclusions, actually strengthens the credibility of your research.