AATtools (version 0.0.3)

aat_covreliability: Compute a dataset's reliability from its covariance matrix

Description

This function computes mean single-difference scores (push minus pull) for individual stimuli, and computes the reliability from that information. Missing values are dealt with using multiple imputation.

This function computes the reliability when stimuli and participants are removed, allowing for the diagnosis of potential sources of unreliability within the data.

Usage

aat_covreliability(
  ds,
  subjvar,
  stimvar,
  pullvar,
  targetvar = NULL,
  rtvar,
  aggfunc = c("mean", "median"),
  algorithm = c("calpha", "lambda2", "lambda4"),
  iters = 5
)
# S3 method for aat_covreliability
print(x, ...)
aat_covreliability_jackknife(
  ds,
  subjvar,
  stimvar,
  pullvar,
  targetvar = NULL,
  rtvar,
  algorithm = c("calpha", "lambda2", "lambda4"),
  iters = 5,
  holdout = c("both", "participant", "stimulus", "cross")
)
# S3 method for aat_covreliability_jackknife
print(x, ...)
# S3 method for aat_covreliability_jackknife
plot(x, ...)

Value

Returns an aat_covreliability object containing the reliability value as well as the dataset and covariance matrix with replaced missing values. When the argument targetvar is provided, the output also contains the reliability of the individual stimulus categories and their intercorrelation.

aat_covreliability_jackknife() returns an aat_covreliability_jackknife object, containing jackknife reliability statistics. If argument holdout was set to "cross", then these statistics are provided in a matrix where rows represent participants and columns represent stimuli. Otherwise, they are provided in data.frames where the stimulus or participant is represented in a column alongside the associated reliability value.

Arguments

ds

the data.frame to use

subjvar

Name of the subject-identifying variable

stimvar

Name of the stimulus-identifying variable

pullvar

Name of the movement-direction identifying variable

targetvar

Optional. Name of the stimulus-category identifying variable

rtvar

Name of the reaction-time identifying variable

aggfunc

The function with which to aggregate the RTs before computing difference scores. Defaults to mean but can be changed to median.

algorithm

The reliability formula to use. Defaults to Cronbach's alpha, but Guttman's Lambda-2 is recommended instead.

iters

If there are missing values (which is almost inevitable) then multiple imputation will be used to complete the covariance matrix - this option sets the number of multiple imputations to be used.

x

Object to be printed

...

Ignored

holdout

What should be removed from the data for computation of jackknife statistics? "both" computes reliability when stimuli and participants are separately removed, while "cross" computes reliability when stimuli and participants are simultaneously removed.

Methods (by generic)

• print(aat_covreliability): Print an aat_covreliability object

Functions

• print(aat_covreliability_jackknife): Print an aat_covreliability_jackknife object

• plot(aat_covreliability_jackknife): Plot an aat_covreliability_jackknife object

Details

When only one stimulus category is indicated, one of the commonly known reliability algorithms provided with the algorithm argument is used. When two stimulus categories are indicated, this function uses Lord's (1963) algorithm to compute the reliability of a double mean difference score, using the algorithms in algorithm to estimate the reliability of indiviau lstimulus categories.

When one wants to compute the reliability of a double median difference score or D-score, aat_splithalf() is recommended instead.

References

Lord, F.Y. (1963), "Elementary Models for Measuring Change", in Problems in Measuring Change, C.W. Harris, ed.. Madison. Wisconsin: University of Wisconsin.

Examples

#We generate a dataset with 16 stimuli in each category
ds<-aat_simulate(biasfx_jitter=40,nstims=16)
ds$stim<-paste0(ds$stim,"-",ds$is_target)

# If Lord's formula and
# bootstrapped splithalf measure something similar,
# then the outcomes should be close to each other.
aat_covreliability(ds=ds,subjvar="subj",stimvar="stim",pullvar="is_pull",
                           targetvar="is_target",rtvar="rt")
aat_splithalf(ds=ds,subjvar="subj",pullvar="is_pull",targetvar="is_target",rtvar="rt",
              algorithm="aat_doublemeandiff",iters=100,plot=FALSE)

#Testing reliability for single-difference scores
ds<-ds[ds$is_target==1,]
aat_covreliability(ds=ds,subjvar="subj",stimvar="stim",pullvar="is_pull",rtvar="rt")
hh<-aat_simulate()
test<-aat_covreliability_jackknife(ds=hh,subjvar="subj",stimvar="stim",pullvar="is_pull",
                                   targetvar="is_target",rtvar="rt",holdout="cross")
print(test)
plot(test)