Several coefficients for factor-analysis reliability have been termed
"omega", which Cho (2021) argues is a misleading misnomer and argues for
using \(\rho\) to represent them all, differentiated by descriptive
subscripts. In our package, we number \(\omega\) based on commonly
applied calculations.
Bentler (1968) first introduced factor-analysis reliability for a
unidimensional factor model with congeneric indicators, labeling the
coeficients \(\alpha\). McDonald (1999) later referred to this
and other reliability coefficients, first as \(\theta\) (in 1970),
then as \(\omega\), which is a source of confusion when reporting
coefficients (Cho, 2021). Coefficients based on factor models were later
generalized to account for multidimenisionality (possibly with
cross-loadings) and correlated errors. The general \(\omega\) formula
implemented in this function is:
$$ \omega = \frac{\left( \sum^{k}_{i = 1} \lambda_i \right)^{2}
Var\left( \psi \right)}{\bold{1}^\prime \hat{\Sigma} \bold{1}}, $$
where \(\hat{\Sigma}\) can be the model-implied covariance matrix from
either the saturated model (i.e., the "observed" covariance matrix, used by
default) or from the hypothesized CFA model, controlled by the
obs.var argument. A \(k\)-dimensional vector \(\bold{1}\) is used
to sum elements in the matrix. Note that if the model includes any directed
effects (latent regression slopes), all coefficients are calculated
from total factor variances:
lavInspect(object, "cov.lv").
Assuming (essential) tau-equivalence (tau.eq=TRUE) makes \(\omega\)
equivalent to coefficient \(\alpha\) from classical test theory
(Cronbach, 1951):
$$ \alpha = \frac{k}{k - 1}\left[ 1 - \frac{\sum^{k}_{i = 1}
\sigma_{ii}}{\sum^{k}_{i = 1} \sigma_{ii} + 2\sum_{i < j} \sigma_{ij}}
\right],$$
where \(k\) is the number of items in a factor's composite,
\(\sigma_{ii}\) signifies item i's variance, and \(\sigma_{ij}\)
signifies the covariance between items i and j. Again, the
obs.var argument controls whether \(\alpha\) is calculated using
the observed or model-implied covariance matrix.
By setting return.total=TRUE, one can estimate reliability for a
single composite calculated using all indicators in a multidimensional
CFA (Bentler, 1972, 2009). Setting return.total = -1 will return
only the total-composite reliability (not per factor).
Higher-Order Factors:
The reliability of a composite that represents a higher-order construct
requires partitioning the model-implied factor covariance matrix \(\Phi\)
in order to isolate the common-factor variance associated only with the
higher-order factor. Using a second-order factor model, the model-implied
covariance matrix of observed indicators \(\hat{\Sigma}\) can be
partitioned into 3 sources:
the second-order common-factor (co)variance:
\(\Lambda \bold{B} \Phi_2 \bold{B}^{\prime} \Lambda^{\prime}\)
the residual variance of the first-order common factors (i.e., not
accounted for by the second-order factor):
\(\Lambda \Psi_{u} \Lambda^{\prime}\)
the measurement error of observed indicators: \(\Theta\)
where \(\Lambda\) contains first-order factor loadings, \(\bold{B}\)
contains second-order factor loadings, \(\Phi_2\) is the model-implied
covariance matrix of the second-order factor(s), and \(\Psi_{u}\) is the
covariance matrix of first-order factor disturbances. In practice, we can
use the full \(\bold{B}\) matrix and full model-implied \(\Phi\) matrix
(i.e., including all latent factors) because the zeros in \(\bold{B}\)
will cancel out unwanted components of \(\Phi\). Thus, we can calculate
the proportion of variance of a composite score calculated from the observed
indicators (e.g., a total score or scale mean) that is attributable to the
second-order factor (i.e., coefficient \(\omega\)):
$$\omega_{L1}=\frac{\bold{1}^{\prime} \Lambda \bold{B} \Phi \bold{B}^{\prime}
\Lambda^{\prime} \bold{1} }{ \bold{1}^{\prime} \hat{\Sigma} \bold{1}}, $$
where \(\bold{1}\) is the k-dimensional vector of 1s and k
is the number of observed indicators in the composite. Note that a
higher-order factor can also have observed indicators.
Categorical Indicators:
When all indicators (per composite) are ordinal, the ord.scale
argument controls whether the coefficient is calculated on the
latent-response scale (FALSE) or on the observed ordinal scale
(TRUE, the default). For \(\omega\)-type coefficients
(tau.eq=FALSE), Green and Yang's (2009, formula 21) approach is used
to transform factor-model results back to the ordinal response scale.
When ord.scale=TRUE, coefficient \(\alpha\) is calculated using the
covariance matrix calculated from the integer-valued numeric weights for
ordinal categories, consistent with its definition (Chalmers, 2018) and the
alpha function in the psych package; this implies
obs.var=TRUE, so obs.var=FALSE will be ignored. When
ord.scale=FALSE, the standard \(\alpha\) formula is applied to the
polychoric correlation matrix ("ordinal \(\alpha\)"; Zumbo et al., 2007),
estimated from the saturated or hypothesized model (see obs.var),
and \(\omega\) is calculated from CFA results without applying Green and
Yang's (2009) correction (see Zumbo & Kroc's, 2019, for a rationalization).
No method has been proposed for calculating reliability with a mixture of
categorical and continuous indicators, so an error is returned if
object includes factors with a mixture of indicator types (unless
omitted using omit.factors). If categorical indicators load on a
different factor(s) than continuous indicators, then reliability will still
be calculated separately for those factors, but return.total must be
FALSE (unless omit.factors is used to isolate factors with
indicators of the same type).
Multilevel Measurement Models:
Under the default settings, compRelSEM() will apply the same formula
in each "block" (group and/or level of analysis). In the case of multilevel
SEMs, this yields "reliability" for latent within- and between-level
components, as proposed by Geldhof et al. (2014). This is not recommended
because the coefficients do not correspond to actual composites that would
be calculated from the observed data. Lai (2021) proposed coefficients for
reliability of actual composites, depending on the type of construct, which
requires specifying the names of constructs for which reliability is desired
(or multiple constructs whose indicators would compose a multidimensional
composite). Configural (config=) and/or shared= constructs
can be specified; the same construct can be specified in both arguments, so
that overall scale-reliability can be estimated for a shared construct by
including it in config. Instead of organizing the output by block
(the default), specifying config= and/or shared= will prompt
organizing the output by $config and/or $shared.
The overall (_2L) scale reliability for configural
constructs is returned, along with the reliability of a purely
individual-level composite (_W, calculated by cluster-mean
centering).
The reliability for a shared construct quantifies
generalizability across both indicators and raters (i.e., subjects rating
their cluster's construct). L<U+00FC>dtke et al. (2011) refer to these as
measurement error and sampling error, respectively. An interrater
reliability (IRR) coefficient is also returned, quantifying
generalizability across rater/sampling-error only. To obtain a
scale-reliability coefficient (quantifying a shared construct's
generalizability across indicator/measurement-error only), include the
same factor name in config=. Jak et al. (2021) recommended
modeling components of the same construct at both levels, but users may
also saturate the within-level model (Lai, 2021).
Be careful about including Level-2 variables in the model, especially
whether it makes sense to include them in a total composite for a Level-2
construct. dropSingle=TRUE only prevents estimating reliability for
a single-indicator construct, not from including such an indicator in a
total composite. It is permissible for shared= constructs to have
indicators at Level-2 only. If it is necessary to model other Level-2
variables (e.g., to justify the missing-at-random assumption when using
missing = "FIML" estimation), they should be placed in the
omit.indicators= argument to exclude them from total composites.