rel (version 1.4.2)

spi: Scott's pi and Fleiss' kappa

Description

Calculates Scott's pi as an index of agreement for two observations of nominal or ordinal scale data, or Fleiss' kappa as an index of agreement for more than two observations of nominal scale data.

Usage

spi(data = NULL, weight = c("unweighted","linear","quadratic"), conf.level = 0.95)

Arguments

data

A matrix with n subjects and m observations (n*m matrix)

weight

A character string specifying "unweighted", "linear", or "quadratic", or a numeric categories*categories matrix with custom weights (see details).

conf.level

Confidence level of the interval.

Value

method

Analysis name

obs

Number of observations

sample

Sample size

est

Point estimate

se

Standard error

lb

Lower confidence boundary

ub

Upper confidence boundary

cont.table

contingency table

data

analyzed data

Details

Scott's pi measures the chance-corrected agreement for two observations (Scott, 1955), and Fleiss' kappa is a generalization of Scott's pi for m observations (Fleiss, 1971). The standard error for Fleiss' kappa is based on the formula from Fleiss et al., 1979. Weights are based on weighted dissimilarities (diagonal = 1, off-diagonal < 1). Linear weights decrease equally with distance from the diagonal and quadratic weights decrease exponentially with distance from the diagonal. Custom weights should be specified as a categories*categories matrix with values <= 1. Incomplete cases are omitted listwise and the confidence interval is based on a t distribution.

References

Scott W. A. (1955). Reliability of content analysis: The case of nominal scale coding. Public Opinion Quarterly, 19(3), 321-325. Fleiss J. L. (1971). Measuring nominal scale agreement among many raters. Psychological Bulletin, 76(5), 378-382. Fleiss J. L., Nee J. C. M., Landis J. R. (1979). Large sample variance of kappa in the case of different sets of raters. Psychological Bulletin, 86(5), 974-977.

Examples

# NOT RUN {
#Sample data: 200 subjects and one 5-category item.
data <- cbind(sample(1:5,200, replace=TRUE),sample(1:5,200, replace=TRUE))

#A numeric categories*categories matrix with custom weights 
cw <- diag(ncol(matrix(0,5,5)))
cw[cw!=diag(cw)] <- runif(20,0,1)

#Scott's pi
spi(data=data, weight="unweighted", conf.level = 0.95)

#Weighted pi with custom weights
spi(data=data, weight=cw, conf.level = 0.95)
# }