Calculate the reliability values (coefficient omega) of a second-order factor
reliabilityL2(object, secondFactor, omit.imps = c("no.conv", "no.se"))The name of a single second-order factor in the
model fitted in object. The function must be called multiple
times to estimate reliability for each higher-order factor.
character vector specifying criteria for omitting
imputations from pooled results. Can include any of
c("no.conv", "no.se", "no.npd"), the first 2 of which are the
default setting, which excludes any imputations that did not
converge or for which standard errors could not be computed. The
last option ("no.npd") would exclude any imputations which
yielded a nonpositive definite covariance matrix for observed or
latent variables, which would include any "improper solutions" such
as Heywood cases. NPD solutions are not excluded by default because
they are likely to occur due to sampling error, especially in small
samples. However, gross model misspecification could also cause
NPD solutions, users can compare pooled results with and without
this setting as a sensitivity analysis to see whether some
imputations warrant further investigation.
Reliability values at Levels 1 and 2 of the second-order factor, as well as the partial reliability value at Level 1
The first formula of the coefficient omega (in the
reliability) will be mainly used in the calculation. The
model-implied covariance matrix of a second-order factor model can be
separated into three sources: the second-order common-factor variance,
the residual variance of the first-order common factors (i.e., not
accounted for by the second-order factor), and the measurement error of
observed indicators:
$$ \hat{\Sigma} = \Lambda \bold{B} \Phi_2 \bold{B}^{\prime} \Lambda^{\prime} + \Lambda \Psi_{u} \Lambda^{\prime} + \Theta, $$
where \(\hat{\Sigma}\) is the model-implied covariance matrix, \(\Lambda\) contains first-order factor loadings, \(\bold{B}\) contains second-order factor loadings, \(\Phi_2\) is the covariance matrix of the second-order factor(s), \(\Psi_{u}\) is the covariance matrix of residuals from first-order factors, and \(\Theta\) is the covariance matrix of the measurement errors from observed indicators. Thus, we can calculate the proportion of variance of a composite score calculated from the observed indicators (e.g., a total score or scale mean) that is attributable to the second-order factor, i.e. coefficient omega at Level 1:
$$ \omega_{L1} = \frac{\bold{1}^{\prime} \Lambda \bold{B} \Phi_2 \bold{B}^{\prime} \Lambda^{\prime} \bold{1}}{\bold{1}^{\prime} \Lambda \bold{B} \Phi_2 \bold{B} ^{\prime} \Lambda^{\prime} \bold{1} + \bold{1}^{\prime} \Lambda \Psi_{u} \Lambda^{\prime} \bold{1} + \bold{1}^{\prime} \Theta \bold{1}}, $$
where \(\bold{1}\) is the k-dimensional vector of 1 and k is the number of observed variables.
The model-implied covariance matrix among first-order factors (\(\Phi_1\)) can be calculated as:
$$ \Phi_1 = \bold{B} \Phi_2 \bold{B}^{\prime} + \Psi_{u}, $$
Thus, the proportion of variance among first-order common factors that is attributable to the second-order factor (i.e., coefficient omega at Level 2) can be calculated as:
$$ \omega_{L2} = \frac{\bold{1_F}^{\prime} \bold{B} \Phi_2 \bold{B}^{\prime} \bold{1_F}}{\bold{1_F}^{\prime} \bold{B} \Phi_2 \bold{B}^{\prime} \bold{1_F} + \bold{1_F}^{\prime} \Psi_{u} \bold{1_F}}, $$
where \(\bold{1_F}\) is the F-dimensional vector of 1 and F is the number of first-order factors. This Level-2 omega can be interpreted as an estimate of the reliability of a hypothetical composite calculated from error-free observable variables representing the first-order common factors. This might only be meaningful as a thought experiment.
An additional thought experiment is possible: If the observed indicators contained only the second-order common-factor variance and unsystematic measurement error, then there would be no first-order common factors because their unique variances would be excluded from the observed measures. An estimate of this hypothetical composite reliability can be calculated as the partial coefficient omega at Level 1, or the proportion of observed variance explained by the second-order factor after partialling out the uniqueness from the first-order factors:
$$ \omega_{L1} = \frac{\bold{1}^{\prime} \Lambda \bold{B} \Phi_2 \bold{B}^{\prime} \Lambda^{\prime} \bold{1}}{\bold{1}^{\prime} \Lambda \bold{B} \Phi_2 \bold{B}^{\prime} \Lambda^{\prime} \bold{1} + \bold{1}^{\prime} \Theta \bold{1}}, $$
Note that if the second-order factor has a direct factor loading on some observed variables, the observed variables will be counted as first-order factors, which might not be desirable.
# NOT RUN {
HS.model3 <- ' visual =~ x1 + x2 + x3
textual =~ x4 + x5 + x6
speed =~ x7 + x8 + x9
higher =~ visual + textual + speed'
fit6 <- cfa(HS.model3, data = HolzingerSwineford1939)
reliability(fit6) # Should provide a warning for the endogenous variables
reliabilityL2(fit6, "higher")
# }
Run the code above in your browser using DataLab