Measurement invariance

Multigroup CFA: configural, metric, scalar, and strict invariance; partial invariance; latent mean comparison

Author

Affiliation

George Michaelides

Norwich Business School, University of East Anglia

Published

July 7, 2026

Learning objectives

By the end of this tutorial you will be able to:

explain what measurement invariance is and why it is a prerequisite for comparing latent means or structural paths across groups
fit a multigroup CFA in lavaan and read the per-group output
test the four-level invariance hierarchy (configural, metric, scalar, strict) using group.equal and lavTestLRT()
supplement the likelihood-ratio test with fit-index change criteria ($\Delta$CFI, $\Delta$RMSEA)
diagnose and handle partial invariance by freeing specific non-invariant parameters with group.partial
compare latent means across groups once (partial) scalar invariance is established
report a measurement-invariance analysis in APA format

Setup

Setup code (you have seen this before)

library(haven)
library(lavaan)
library(ggplot2)

We use the same two-factor Positive Affect (PA) / Negative Affect (neg) measurement model introduced in Tutorial 07, fitted separately for males and females. There are 178 males and 154 females in the ps dataset — enough per group for a stable multigroup CFA.

What is measurement invariance?

Measurement invariance means that the same measurement model holds in every group. Concretely: the same items measure the same latent factors, with the same relationships between items and factors, in males and females, or in any other groups you want to compare.

Why it matters

Suppose you find that females score higher than males on a “positive affect” factor. Before you interpret that as a real psychological difference, you must rule out a measurement artefact: maybe the items load differently on the factor in the two groups, so a “unit” of PA is not the same quantity for males and females. Measurement invariance testing establishes whether the measurement model is equivalent across groups. Without it, any group difference in latent means or structural paths could be a difference in what the items measure, not a difference in the construct itself.

Measurement invariance is a prerequisite for group comparison

Before you can compare latent means or structural paths across groups, you must establish that the measurement model is equivalent — that the same items measure the same factors in the same way in both groups. This is what measurement invariance testing does. Without it, observed group differences could be due to measurement artefacts rather than true group differences. We introduced this idea in Tutorial 16; here we develop it fully.

The four levels

Measurement invariance is tested as a hierarchy: you start with the most liberal model and progressively constrain parameters to be equal across groups, checking at each step whether the constraint damages fit.

Level	Also called	What is constrained equal	What it licenses
Configural	pattern	Same factor structure (same items on same factors)	Comparing model form across groups
Metric	weak	Factor loadings	Comparing structural paths / variances across groups
Scalar	strong	Loadings + item intercepts	Comparing latent means across groups
Strict	strict	Loadings + intercepts + residual variances	Comparing observed score variances

Each level nests inside the one above it: scalar invariance implies metric invariance implies configural invariance. You test them in order, and you stop when a constraint clearly damages fit.

The measurement-invariance hierarchy. Each level adds equality constraints and licenses a stronger kind of cross-group comparison.

Specifying a multigroup model in lavaan

The measurement model is the two-factor CFA from Tutorial 07:

model_cfa <- '
  PA  =~ pa1 + pa2 + pa3
  neg =~ na1 + na2 + na3
  PA  ~~ neg
'

To fit it for both groups at once, pass the group argument to cfa():

fit_config <- cfa(model_cfa, data = ps, group = "gender", estimator = "MLR")
summary(fit_config, standardized = TRUE, fit.measures = TRUE)

## lavaan 0.6-21 ended normally after 37 iterations
## 
##   Estimator                                         ML
##   Optimization method                           NLMINB
##   Number of model parameters                        38
## 
##   Number of observations per group:               Used       Total
##     Female                                         152         154
##     Male                                           178         178
## 
## Model Test User Model:
##                                               Standard      Scaled
##   Test Statistic                                17.150      17.710
##   Degrees of freedom                                16          16
##   P-value (Chi-square)                           0.376       0.341
##   Scaling correction factor                                  0.968
##     Yuan-Bentler correction (Mplus variant)                       
##   Test statistic for each group:
##     Female                                       6.244       6.244
##     Male                                        11.466      11.466
## 
## Model Test Baseline Model:
## 
##   Test statistic                              1020.405     987.924
##   Degrees of freedom                                30          30
##   P-value                                        0.000       0.000
##   Scaling correction factor                                  1.033
## 
## User Model versus Baseline Model:
## 
##   Comparative Fit Index (CFI)                    0.999       0.998
##   Tucker-Lewis Index (TLI)                       0.998       0.997
##                                                                   
##   Robust Comparative Fit Index (CFI)                         0.998
##   Robust Tucker-Lewis Index (TLI)                            0.997
## 
## Loglikelihood and Information Criteria:
## 
##   Loglikelihood user model (H0)              -2594.689   -2594.689
##   Scaling correction factor                                  0.963
##       for the MLR correction                                      
##   Loglikelihood unrestricted model (H1)      -2586.114   -2586.114
##   Scaling correction factor                                  0.965
##       for the MLR correction                                      
##                                                                   
##   Akaike (AIC)                                5265.378    5265.378
##   Bayesian (BIC)                              5409.744    5409.744
##   Sample-size adjusted Bayesian (SABIC)       5289.207    5289.207
## 
## Root Mean Square Error of Approximation:
## 
##   RMSEA                                          0.021       0.025
##   90 Percent confidence interval - lower         0.000       0.000
##   90 Percent confidence interval - upper         0.076       0.079
##   P-value H_0: RMSEA <= 0.050                    0.747       0.714
##   P-value H_0: RMSEA >= 0.080                    0.037       0.047
##                                                                   
##   Robust RMSEA                                               0.025
##   90 Percent confidence interval - lower                     0.000
##   90 Percent confidence interval - upper                     0.077
##   P-value H_0: Robust RMSEA <= 0.050                         0.728
##   P-value H_0: Robust RMSEA >= 0.080                         0.039
## 
## Standardized Root Mean Square Residual:
## 
##   SRMR                                           0.021       0.021
## 
## Parameter Estimates:
## 
##   Standard errors                             Sandwich
##   Information bread                           Observed
##   Observed information based on                Hessian
## 
## 
## Group 1 [Female]:
## 
## Latent Variables:
##                    Estimate  Std.Err  z-value  P(>|z|)   Std.lv  Std.all
##   PA =~                                                                 
##     pa1               1.000                               1.098    0.879
##     pa2               0.949    0.076   12.544    0.000    1.041    0.789
##     pa3               0.941    0.072   13.128    0.000    1.033    0.843
##   neg =~                                                                
##     na1               1.000                               0.901    0.847
##     na2               1.073    0.085   12.552    0.000    0.967    0.872
##     na3               1.026    0.088   11.699    0.000    0.925    0.808
## 
## Covariances:
##                    Estimate  Std.Err  z-value  P(>|z|)   Std.lv  Std.all
##   PA ~~                                                                 
##     neg              -0.429    0.111   -3.857    0.000   -0.433   -0.433
## 
## Intercepts:
##                    Estimate  Std.Err  z-value  P(>|z|)   Std.lv  Std.all
##    .pa1               3.421    0.101   33.774    0.000    3.421    2.739
##    .pa2               3.276    0.107   30.627    0.000    3.276    2.484
##    .pa3               3.309    0.099   33.278    0.000    3.309    2.699
##    .na1               2.895    0.086   33.519    0.000    2.895    2.719
##    .na2               2.796    0.090   31.106    0.000    2.796    2.523
##    .na3               2.638    0.093   28.419    0.000    2.638    2.305
## 
## Variances:
##                    Estimate  Std.Err  z-value  P(>|z|)   Std.lv  Std.all
##    .pa1               0.355    0.075    4.762    0.000    0.355    0.228
##    .pa2               0.655    0.105    6.214    0.000    0.655    0.377
##    .pa3               0.435    0.080    5.429    0.000    0.435    0.290
##    .na1               0.321    0.058    5.559    0.000    0.321    0.283
##    .na2               0.293    0.067    4.379    0.000    0.293    0.239
##    .na3               0.455    0.067    6.817    0.000    0.455    0.347
##     PA                1.205    0.157    7.697    0.000    1.000    1.000
##     neg               0.812    0.115    7.035    0.000    1.000    1.000
## 
## 
## Group 2 [Male]:
## 
## Latent Variables:
##                    Estimate  Std.Err  z-value  P(>|z|)   Std.lv  Std.all
##   PA =~                                                                 
##     pa1               1.000                               0.829    0.774
##     pa2               1.076    0.105   10.229    0.000    0.892    0.729
##     pa3               1.288    0.125   10.272    0.000    1.067    0.871
##   neg =~                                                                
##     na1               1.000                               0.982    0.852
##     na2               0.828    0.076   10.955    0.000    0.814    0.771
##     na3               0.958    0.071   13.502    0.000    0.941    0.864
## 
## Covariances:
##                    Estimate  Std.Err  z-value  P(>|z|)   Std.lv  Std.all
##   PA ~~                                                                 
##     neg              -0.346    0.084   -4.109    0.000   -0.426   -0.426
## 
## Intercepts:
##                    Estimate  Std.Err  z-value  P(>|z|)   Std.lv  Std.all
##    .pa1               3.820    0.080   47.581    0.000    3.820    3.566
##    .pa2               3.500    0.092   38.163    0.000    3.500    2.860
##    .pa3               3.590    0.092   39.102    0.000    3.590    2.931
##    .na1               2.685    0.086   31.090    0.000    2.685    2.330
##    .na2               2.652    0.079   33.509    0.000    2.652    2.512
##    .na3               2.590    0.082   31.732    0.000    2.590    2.378
## 
## Variances:
##                    Estimate  Std.Err  z-value  P(>|z|)   Std.lv  Std.all
##    .pa1               0.461    0.066    6.964    0.000    0.461    0.402
##    .pa2               0.702    0.095    7.405    0.000    0.702    0.469
##    .pa3               0.361    0.096    3.773    0.000    0.361    0.241
##    .na1               0.364    0.070    5.224    0.000    0.364    0.274
##    .na2               0.453    0.067    6.732    0.000    0.453    0.406
##    .na3               0.301    0.057    5.238    0.000    0.301    0.254
##     PA                0.687    0.119    5.755    0.000    1.000    1.000
##     neg               0.964    0.126    7.680    0.000    1.000    1.000

By default lavaan fits the configural invariance model — the same factor structure in both groups, but all parameters (loadings, intercepts, residual variances, factor variances/covariances) freely estimated in each group. The output shows a separate parameter block for each group. The fit is good ($\chi^2(16) = 17.15$, $p = .376$, CFI = .999, RMSEA = .021), so the same two-factor structure is tenable in both males and females. Configural invariance holds.

Testing the invariance hierarchy

To test whether the parameters are equivalent across groups, progressively constrain them with the group.equal argument and compare each model to the previous one with lavTestLRT() (the scaled likelihood-ratio test for MLR models — see Tutorial 13).

Metric (weak) invariance — equal loadings

# Metric invariance: constrain all loadings equal across groups
fit_metric <- cfa(model_cfa, data = ps, group = "gender",
                  group.equal = "loadings", estimator = "MLR")

lavTestLRT(fit_config, fit_metric)

## 
## Scaled Chi-Squared Difference Test (method = "satorra.bentler.2001")
## 
## lavaan->lavTestLRT():  
##    lavaan NOTE: The "Chisq" column contains standard test statistics, not the 
##    robust test that should be reported per model. A robust difference test is 
##    a function of two standard (not robust) statistics.
## 
##            Df    AIC    BIC  Chisq Chisq diff   RMSEA Df diff Pr(>Chisq)  
## fit_config 16 5265.4 5409.7 17.150                                        
## fit_metric 20 5268.1 5397.2 27.835     12.714 0.10534       4    0.01276 *
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

The scaled $\chi^2$ difference is significant ($\Delta\chi^2 = 12.71$, $\Delta df = 4$, $p = .013$): by the strict likelihood-ratio test, the loadings are not equivalent across groups, and metric invariance does not hold. Before we decide what to do, compare the fit indices:

rbind(
  configural = fitMeasures(fit_config, c("cfi", "rmsea")),
  metric     = fitMeasures(fit_metric, c("cfi", "rmsea"))
)

##                  cfi      rmsea
## configural 0.9988387 0.02087302
## metric     0.9920891 0.04872611

CFI drops from .999 to .992 ($\Delta$CFI = .007) and RMSEA rises from .021 to .049 ($\Delta$RMSEA = .028).

Supplementing the LRT with fit-index change criteria

With anything but the smallest samples, the $\chi^2$ difference test is overpowered — it flags as “significant” constraints that differ only trivially. A widely used supplement is change in fit indices. Chen (2007)’s simulation-based criteria, for two groups and $N$ in the hundreds, are:

Index	Criterion for invariance to hold
$\Delta$CFI	$\leq .010$
$\Delta$RMSEA	$\leq .015$

The LRT and the fit-index criteria can disagree

Here they do. The LRT says metric invariance fails ($p = .013$). But $\Delta$CFI = .007 is below the .010 threshold, while $\Delta$RMSEA = .028 exceeds the .015 threshold. The two indices disagree. This is common in real data and is exactly the situation partial invariance analysis was designed for: rather than abandon metric invariance, we ask which loadings differ and free just those.

Partial invariance

Partial invariance holds when most, but not all, of a parameter type is equivalent across groups — for example, when most loadings are equal but one or two differ. Byrne et al. (1989) argued that as long as at least two loadings per factor (the marker plus at least one other) are invariant, the factor’s scale is comparable across groups and subsequent comparisons are interpretable.

Identifying the non-invariant parameters

The correct lavaan tool for finding equality constraints to release is lavTestScore(), which gives a univariate score test for each constrained parameter:

lavTestScore(fit_metric)$uni

## 
## univariate score tests:
## 
##    lhs op   rhs    X2 df p.value
## 1 .p2. == .p25. 0.057  1   0.812
## 2 .p3. == .p26. 4.875  1   0.027
## 3 .p5. == .p28. 4.474  1   0.034
## 4 .p6. == .p29. 0.300  1   0.584

The two largest univariate score tests (both $p < .05$) flag two of the four freely estimated loadings. We can confirm which by looking at the configural model’s loadings by group:

pe <- parameterEstimates(fit_config, standardized = TRUE)
pe[pe$op == "=~", c("lhs", "rhs", "group", "est", "std.all")]

##    lhs rhs group   est std.all
## 1   PA pa1     1 1.000   0.879
## 2   PA pa2     1 0.949   0.789
## 3   PA pa3     1 0.941   0.843
## 4  neg na1     1 1.000   0.847
## 5  neg na2     1 1.073   0.872
## 6  neg na3     1 1.026   0.808
## 24  PA pa1     2 1.000   0.774
## 25  PA pa2     2 1.076   0.729
## 26  PA pa3     2 1.288   0.871
## 27 neg na1     2 1.000   0.852
## 28 neg na2     2 0.828   0.771
## 29 neg na3     2 0.958   0.864

The marker indicators (pa1, na1) are fixed to 1 in both groups by default, so they cannot differ. Of the free loadings, pa3 (0.941 in females vs 1.288 in males) and na2 (1.073 vs 0.828) differ the most — the same two parameters the score test flagged.

Fitting a partial metric model

Free specific parameters with group.partial while keeping the rest constrained:

# Partial metric invariance: loadings equal except for pa3 and na2
fit_partial <- cfa(model_cfa, data = ps, group = "gender",
                   group.equal = "loadings",
                   group.partial = c("PA =~ pa3", "neg =~ na2"),
                   estimator = "MLR")

lavTestLRT(fit_config, fit_partial)

## 
## Scaled Chi-Squared Difference Test (method = "satorra.bentler.2001")
## 
## lavaan->lavTestLRT():  
##    lavaan NOTE: The "Chisq" column contains standard test statistics, not the 
##    robust test that should be reported per model. A robust difference test is 
##    a function of two standard (not robust) statistics.
## 
##             Df    AIC    BIC  Chisq Chisq diff RMSEA Df diff Pr(>Chisq)
## fit_config  16 5265.4 5409.7 17.150                                    
## fit_partial 18 5262.5 5399.3 18.287     1.3876     0       2     0.4997

Freeing just two loadings makes the model fit as well as the configural model ($\Delta\chi^2 = 1.39$, $\Delta df = 2$, $p = .500$; $\Delta$CFI $\approx .0001$). Partial metric invariance holds. The PA and neg factors still have at least two invariant loadings each (pa1/pa2 and na1/na3), so their scales remain comparable across gender.

group.partial frees in both groups

group.partial = c("PA =~ pa3", "neg =~ na2") releases the equality constraint on those loadings — they are freely estimated in both groups, while all other loadings remain equal. You do not need to specify group-specific labels unless you want to constrain two parameters equal within a group.

Scalar (strong) invariance — adding equal intercepts

With partial metric invariance established, test whether the item intercepts are also equal:

# Partial scalar invariance: equal loadings (except pa3, na2) + equal intercepts
fit_scalar <- cfa(model_cfa, data = ps, group = "gender",
                  group.equal = c("loadings", "intercepts"),
                  group.partial = c("PA =~ pa3", "neg =~ na2"),
                  estimator = "MLR")

lavTestLRT(fit_partial, fit_scalar)

## 
## Scaled Chi-Squared Difference Test (method = "satorra.bentler.2001")
## 
## lavaan->lavTestLRT():  
##    lavaan NOTE: The "Chisq" column contains standard test statistics, not the 
##    robust test that should be reported per model. A robust difference test is 
##    a function of two standard (not robust) statistics.
## 
##             Df    AIC    BIC  Chisq Chisq diff   RMSEA Df diff Pr(>Chisq)
## fit_partial 18 5262.5 5399.3 18.287                                      
## fit_scalar  22 5261.4 5383.0 25.165     6.8986 0.06617       4     0.1413

The intercept constraints do not damage fit ($\Delta\chi^2 = 6.90$, $\Delta df = 4$, $p = .141$; $\Delta$CFI = .003). Partial scalar invariance holds. This is the level required to compare latent means across groups.

Strict invariance (optional)

Adding equal residual variances tests strict invariance. This level is rarely required for substantive conclusions — scalar invariance is enough for comparing latent means — so we test it only for completeness:

fit_strict <- cfa(model_cfa, data = ps, group = "gender",
                  group.equal = c("loadings", "intercepts", "residuals"),
                  group.partial = c("PA =~ pa3", "neg =~ na2"),
                  estimator = "MLR")

lavTestLRT(fit_scalar, fit_strict)

## 
## Scaled Chi-Squared Difference Test (method = "satorra.bentler.2001")
## 
## lavaan->lavTestLRT():  
##    lavaan NOTE: The "Chisq" column contains standard test statistics, not the 
##    robust test that should be reported per model. A robust difference test is 
##    a function of two standard (not robust) statistics.
## 
##            Df    AIC    BIC  Chisq Chisq diff    RMSEA Df diff Pr(>Chisq)
## fit_scalar 22 5261.4 5383.0 25.165                                       
## fit_strict 28 5257.4 5356.1 33.128     7.8205 0.043272       6     0.2516

Strict invariance also holds ($p = .252$). In practice you would usually stop at scalar.

Comparing latent means across groups

Once (partial) scalar invariance is established, the factor scale is fixed across groups and latent means become comparable. Refit the partial scalar model with meanstructure = TRUE:

fit_means <- cfa(model_cfa, data = ps, group = "gender",
                 group.equal = c("loadings", "intercepts"),
                 group.partial = c("PA =~ pa3", "neg =~ na2"),
                 estimator = "MLR", meanstructure = TRUE)

# Latent mean of group 1 (Female, alphabetically first) is fixed to 0; group 2 (Male) is estimated
pe_means <- parameterEstimates(fit_means)
pe_means[pe_means$op == "~1" & pe_means$lhs %in% c("PA", "neg"),
         c("lhs", "group", "est", "se", "z", "pvalue")]

##    lhs group    est    se      z pvalue
## 22  PA     1  0.000 0.000     NA     NA
## 23 neg     1  0.000 0.000     NA     NA
## 45  PA     2  0.263 0.115  2.291  0.022
## 46 neg     2 -0.131 0.116 -1.127  0.260

Latent factor means by gender (females fixed at 0 as the alphabetical reference). Males score significantly higher on Positive Affect; the Negative Affect difference is not significant.

Positive Affect: the male latent mean is 0.263 higher than the female mean ($z = 2.29$, $p = .022$). Males report significantly higher PA, in the metric of the pa1 marker indicator.
Negative Affect: the male mean is 0.131 higher than the female mean ($z = 1.13$, $p = .260$) — not significantly different from females.

Why the reference group mean is zero

In multigroup lavaan, the first group’s latent mean is fixed to 0 (identification), and the other group’s latent mean is estimated as a difference from that reference. The signs and $p$-values are therefore group-order dependent in form but not in substance — reversing the group order flips the sign of the estimate and leaves the $p$-value unchanged.

Note that lavaan orders groups alphabetically by their label values, not by the factor() levels you declared. Although ps$gender was declared with Male as level 1, the labels “Female” and “Male” sort Female first alphabetically, so Female is the reference group (Group 1) and Male (Group 2) is estimated. To force Male as the reference regardless of label order, pass group.label = c("Male", "Female") in the cfa() call (lavaan ≥ 0.6-19).

Reporting measurement invariance

An APA-style summary of the analysis above:

To establish the measurement equivalence of the Positive Affect and Negative Affect scales across gender, we tested a sequence of multigroup confirmatory factor models using robust maximum likelihood (MLR). The configural model fitted well, $\chi^2(16) = 17.15$, $p = .376$, CFI = .999, RMSEA = .021, establishing that the same two-factor structure held in both groups. Imposing equal loadings significantly worsened fit according to the scaled chi-square difference test, $\Delta\chi^2_{\text{SB}}(4) = 12.71$, $p = .013$, although the change in CFI (.007) was below the .010 criterion recommended by Chen (2007). Score tests identified two loadings (pa3, na2) as non-invariant; freeing these yielded partial metric invariance, $\Delta\chi^2_{\text{SB}}(2) = 1.39$, $p = .500$. Adding equal intercepts did not worsen fit, $\Delta\chi^2_{\text{SB}}(4) = 6.90$, $p = .141$, establishing partial scalar invariance. Under this model, males reported significantly higher Positive Affect than females (latent mean difference = 0.263, $z = 2.29$, $p = .022$); the Negative Affect difference was not significant (difference = 0.131, $z = 1.13$, $p = .260$).

Try it

Re-run the hierarchy above using age (recoded into two broad bands, e.g. 1–3 vs 4–5) as the grouping variable instead of gender. Does partial scalar invariance hold? If so, do the two age bands differ in latent PA or neg?

Solution

# Collapse age into two bands
ps$age_band <- ifelse(ps$age <= 3, "Younger", "Older")
ps$age_band <- factor(ps$age_band, levels = c("Younger", "Older"))
table(ps$age_band)

## 
## Younger   Older 
##     202     128

fit_cfg_a <- cfa(model_cfa, data = ps, group = "age_band", estimator = "MLR")
fit_met_a <- cfa(model_cfa, data = ps, group = "age_band",
                 group.equal = "loadings", estimator = "MLR")
fit_sca_a <- cfa(model_cfa, data = ps, group = "age_band",
                 group.equal = c("loadings", "intercepts"), estimator = "MLR")

lavTestLRT(fit_cfg_a, fit_met_a)

## 
## Scaled Chi-Squared Difference Test (method = "satorra.bentler.2001")
## 
## lavaan->lavTestLRT():  
##    lavaan NOTE: The "Chisq" column contains standard test statistics, not the 
##    robust test that should be reported per model. A robust difference test is 
##    a function of two standard (not robust) statistics.
## 
##           Df    AIC    BIC  Chisq Chisq diff  RMSEA Df diff Pr(>Chisq)  
## fit_cfg_a 16 5268.8 5413.2 33.772                                       
## fit_met_a 20 5268.5 5397.7 41.435     9.0521 0.0805       4    0.05981 .
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

lavTestLRT(fit_met_a, fit_sca_a)

## 
## Scaled Chi-Squared Difference Test (method = "satorra.bentler.2001")
## 
## lavaan->lavTestLRT():  
##    lavaan NOTE: The "Chisq" column contains standard test statistics, not the 
##    robust test that should be reported per model. A robust difference test is 
##    a function of two standard (not robust) statistics.
## 
##           Df    AIC    BIC  Chisq Chisq diff   RMSEA Df diff Pr(>Chisq)  
## fit_met_a 20 5268.5 5397.7 41.435                                        
## fit_sca_a 24 5269.9 5383.9 50.819     9.7171 0.09146       4    0.04547 *
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

# If scalar holds, compare latent means
fit_means_a <- cfa(model_cfa, data = ps, group = "age_band",
                   group.equal = c("loadings", "intercepts"),
                   estimator = "MLR", meanstructure = TRUE)
pe_a <- parameterEstimates(fit_means_a)
pe_a[pe_a$op == "~1" & pe_a$lhs %in% c("PA", "neg"),
     c("lhs", "group", "est", "se", "z", "pvalue")]

##    lhs group    est    se      z pvalue
## 22  PA     1  0.000 0.000     NA     NA
## 23 neg     1  0.000 0.000     NA     NA
## 45  PA     2 -0.208 0.116 -1.793  0.073
## 46 neg     2  0.079 0.115  0.683  0.494

If the lavTestLRT(fit_cfg_a, fit_met_a) result is non-significant, full metric invariance holds and you can skip partial invariance. If it is significant, run lavTestScore(fit_met_a)$uni and free the flagged loadings with group.partial, as we did for gender.

Common errors

Comparing latent means without scalar invariance

Latent mean differences are only meaningful if the factor scale is fixed across groups, which requires (partial) scalar invariance. If you compare latent means after only configural or metric invariance, a group difference could reflect different item intercepts — i.e. different baseline response levels — rather than a real latent mean difference. Always establish scalar invariance first.

Abandoning the hierarchy too early

If metric invariance fails, do not jump straight to “the factor is not comparable across groups.” First check the $\Delta$CFI / $\Delta$RMSEA criteria — the LRT is overpowered at typical $N$ — and then run lavTestScore() to see whether the failure is driven by one or two items. Partial invariance often rescues an otherwise usable model.

Comparing structural paths without measurement invariance

If you compare structural paths across groups (the multigroup-SEM moderation strategy in Tutorial 16) without first establishing metric invariance, any path differences you find could be artefacts of the items measuring the factors differently in the two groups. Always establish at least (partial) metric invariance before comparing structural paths.

Insufficient sample size per group

Multigroup CFA needs enough participants per group to estimate the model reliably. A rough rule of thumb is 100+ per group. With smaller groups, models may fail to converge or produce unstable estimates. The PS dataset’s 178 males and 154 females are adequate; its five age bands (the smallest has 21) are not — collapse bands or use a different grouping variable if a group is too small.

Summary & next tutorial

You have now seen how to establish that a measurement model means the same thing in different groups — the prerequisite for any cross-group comparison of latent means or structural paths. The key ideas:

Test the hierarchy in order: configural $\rightarrow$ metric $\rightarrow$ scalar $\rightarrow$ (optional) strict.
Use lavTestLRT() for the scaled $\chi^2$ difference test, and check $\Delta$CFI / $\Delta$RMSEA against Chen’s (2007) criteria — the two can disagree, especially with larger $N$.
When full invariance fails, use lavTestScore() to identify the culprits and group.partial to free them — partial invariance is usually enough.
Compare latent means only after (partial) scalar invariance is established, using meanstructure = TRUE.

For multigroup SEM as a moderation strategy — testing whether structural paths differ across groups once measurement invariance is established — see Tutorial 16. For reporting the results, see Tutorial 13.

References

Byrne, B. M., Shavelson, R. J., & Muthén, B. (1989). Testing for the equivalence of factor covariance and mean structures: The issue of partial measurement invariance. Psychological Bulletin, 105(3), 456–466. https://doi.org/10.1037/0033-2909.105.3.456

Chen, F. F. (2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 14(3), 464–504. https://doi.org/10.1080/10705510701301834

Hoyle, R. H. (2012). Handbook of structural equation modeling.

Putnick, D. L., & Bornstein, M. H. (2016). Measurement invariance conventions and reporting: The state of the art and future directions for psychological research. Developmental Review, 41, 71–90. https://doi.org/10.1016/j.dr.2016.06.004

Rosseel, Y. (2012). Lavaan: An r package for structural equation modeling. Journal of Statistical Software, 48(2), 1–36. https://doi.org/10.18637/jss.v048.i02

Vandenberg, R. J., & Lance, C. E. (2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3(1), 4–70. https://doi.org/10.1177/109442810031002

Index	Criterion for invariance to hold
\(\Delta\)CFI	\(\leq .010\)
\(\Delta\)RMSEA	\(\leq .015\)