psych (version 2.1.6)

reliability: Reports 7 different estimates of scale reliabity including alpha, omega, split half

Description

Revelle and Condon, (2019) reviewed the problem of reliability in a tutorial meant to useful to the theoretician as well as the practitioner. Although there are a number of functions in psych for estimating reliability of single scales, (e.g. alpha and omega), for split half reliability splitHalf or for finding test-retest reliability testRetest or multilevel reliability mlr, the reliability function combines several of these functions to report these recommended measures for multiple scales.

To quote from Revelle and Condon (2019) ``Reliability is a fundamental problem for measurement in all of science for `(a)ll measurement is befuddled by error' (p 294 McNemar, 1946). Perhaps because psychological measures are more befuddled than those of the other natural sciences, psychologists have long studied the problem of reliability.

``Issues of reliability are fundamental to understanding how correlations between observed variables are (attenuated) underestimates of the relationships between the underlying constructs, how observed estimates of a person's score are biased estimates of their latent score, and how to estimate the confidence intervals around any particular measurement. Understanding the many ways to estimate reliability as well as the ways to use these estimates allows one to better assess individuals and to evaluate selection and prediction techniques. This is not just a problem for measurement specialists but for all who want to make theoretical inferences from observed data.

`` It is no longer acceptable to report one coefficient that is only correct if all items are exactly equally good measures of a construct. Researchers are encouraged to report at least two coefficients (e.g., omega_h and omega_t) and then discuss why each is appropriate for the inference that is being made. They are discouraged from reporting just alpha unless they can justify the assumptions implicit in using it (i.e., tau equivalence and unidimensionality)." Here we make it easy to do so.

Although the alpha and omega functions will find reliability estimates for a single scale, and scoreItems and scoreOverlap will find alpha for multiple scales, it sometimes is convenient to call omega and splitHalf for multiple scales. reliability takes a keys list (suitable for scoreItems ) and then finds hierarchical and total omega as well as split half reliabilities for each separate scale.

plot.reliability takes the output of reliability and displays it as a dot chart showing the values of both omegas as well as alpha and the distributions of split half reliabilities.

Usage

reliability(keys, items, nfactors = 2, split = TRUE, raw=FALSE, plot=FALSE,hist=FALSE)
# S3 method for reliability
plot(x,omega=TRUE,alpha=TRUE,split=TRUE,add=FALSE,xlim=NULL,
     main=NULL,...)

Arguments

keys

A list of items to be scored (may be taken from a keys.list for scoreItems. This list may contain one or more keys. )

items

The matrix or data.frame of items to be scored. Can be substantially greater than the items included in keys. For just those items in each key are scored.

nfactors

Omega is not well defined for two factors, but for small sets of items, two is the better choice. For larger number of items per scale, 3 is probably preferable.

split

By default, find splitHalf reliabilities as well as the omega statistics. When plotting, split implies that raw was called in reliability.

plot

By default, suppress the omega plots for each scale.

raw

If TRUE, return a list of all the possible splits

hist

If TRUE then split and raw are forced to TRUE and the histograms of the split half values are drawn. (Otherwise, just return the values for later plotting)

x

The object returned from reliability

omega

Add in the values of omega_h and omega_t

alpha

Add the value of alpha

add

Allows us to merge this figure with other ones

main

Defaults to "Split half distributions + omega, alpha"

xlim

The xlim of the plot

...

Other graphical parameters

Value

omega_h

Omega_h is the (model based) hierarchical estimate of the general factor saturation of a scale.

alpha

The conventional alpha statistic (which is not model based)

omega.tot

A model based estimate of the total reliability of a scale

max.split

The greatest split half reliability of the scale. Found by finding all possible splits (if this is < 10,000) or sampled from 10,000 possible splits.

min.split

The lowest split half reliability of the scale. An estimate of beta (see iclust).

mean.r

The average correlation of the items in the scale

med.r

The median correlation of the items. If this differs from the mean, that is a sign of poor scale.

splits

A list of the split half values for all possible splits.

Details

reliability is basically just a wrapper for omegah and splitHalf. Revelle and Condon (2019) recommended reporting at least three reliability statistics for any scale, here we make it easy to do.

If the hist option is set to true, histgrams and density plots of the split half values for each test are also shown. The output from reliability can be passed to error.dots to show the reliability statistics for multiple scales graphically. It is however more useful to just call the plot.reliability function to the basic information.

For detailed analysis of any one scale, it is recommended to do a complete omega analysis, perhaps combined with a splitHalf analysis. The reliability function is just meant for the case where the user has multiple scales (perhaps scored using scoreItems) and then wants to get more complete reliability information for all of the scales.

plot.reliability provides a dot chart summary of the distributions of the split half values, as well as the estimates of omega and alpha. It can also be called by just issuing a plot command.

References

Revelle, William. (in prep) An introduction to psychometric theory with applications in R. Springer. Working draft available at https://personality-project.org/r/book/

Revelle, W. and Condon, D.M. (2019) Reliability from alpha to omega: A tutorial. Psychological Assessment, 31, 12, 1395-1411. https://doi.org/10.1037/pas0000754. https://psyarxiv.com/2y3w9/ Preprint available from PsyArxiv

See Also

See Also omega to find more complete output for the various omega analyses,splitHalf to show more detail on split half estimates, scoreItems to find scores on multiple scales using unit weights, testRetest to find test retest reliabilities, mlr to find multilevel reliabilities.

Examples

# NOT RUN {
reliability(psychTools::bfi.keys,psychTools::bfi)

# }
# NOT RUN {
#this takes a few seconds but shows nice graphic displays

spi.rel <- reliability(psychTools::spi.keys,psychTools::spi,hist=TRUE) #graph them
spi.rel #show them

#plot them using plot.reliability
plot(spi.rel)  #draw the density distrbutions

plot(spi.rel,split=FALSE) #don't draw the split half density distribution

#or do this without the densities

#plot the first three values in a dot chart
error.dots(spi.rel$result.df[,1],sort=FALSE, xlim=c(.3,1),head=16,tail=16,
          main = expression(paste(omega[h], ~~~~ alpha,~~~~ omega[t])))
  #plot the omega_h values
error.dots(spi.rel$result.df[,2],sort=FALSE,pch=2,xlim=c(.3,1),head=16,tail=16,
       main="",labels="",add=TRUE)#add the alpha values
error.dots(spi.rel$result.df[,3],sort=FALSE, xlim=c(.3,1),head=16,tail=16, 
       pch=3,labels="", main="",add=TRUE) #and the omega_t values
       
#or, show the smallest and greatest split half, as well as alpha
error.dots(spi.rel$result.df[,4],sort=FALSE, xlim=c(.3,1),head=16,tail=16,
          main = expression(paste(beta, ~~~~ alpha,~~~~  glb)))
error.dots(spi.rel$result.df[,5],sort=FALSE,pch=5,xlim=c(.3,1),head=16,tail=16,
       main="",labels="",add=TRUE)#add the GLB values
error.dots(spi.rel$result.df[,2],sort=FALSE,pch=2,xlim=c(.3,1),head=16,tail=16,
       main="",labels="",add=TRUE)#add the alpha values
       

# }