oconnell: O'Connell-Dobson estimators for multiobserver agreement.

Description

Use the O'Connell-Dobson estimator of agreement for nominal or ordinal data. This includes a range of statistics on agreement for assuming either distinct or homogeneous items.

Usage

oconnell(X, weights=c("unweighted","linear","quadratic"), i=NULL, score = NULL)

Value

X: As input
i: As input
nrater: Number of observers
nscore: Number of categories
nsubj: Number of subjects
p1[j,k]: Probability of observer j giving score k when observers are distinct
p2[k]: Probability of score k when observers are homogeneous
w1[j,k]: Weighted average of d[] for observer j, score k
w2[k]: Weighted average of d[] for score k when observers are homogeneous
d[j]: Amount of disagreement for subject j
s1[j]: Chance-corrected agreement statistic for subject j when observers are distinct
s2[j]: Chance-corrected agreement statistic for subject j when observers are homogeneous; s[j]=1-d[j]/expdel.
delta[j,k]: j<k: amount of disagreement expected by change for observers j and k; j>k amount of disagreement expected by chance for observers j and k when observers are homogeneous
expd1: Amount of disagreement expected by chance in null case when observers are distinct
expd2: Amount of disagreement expected by chance when observers are homogeneous
dbar: Average value of d[] over all subjects
sav1: Chance-corrected agreement statistic over all subjects when observers are distinct
sav2: Chance-corrected agreement statistic over all subjects when observers are homogeneous
var0s1: Null variance of S when observers are distinct
var0s2: Null variance of S when observers are homogeneous
vars1: Unconstrained variance of S when observers are distinct
vars2: Unconstrained variance of S when observers are homogeneous
v0sav1: Null variance of Sav when observers are distinct
v0sav2: Null variance of Sav when observers are homogeneous
vsav1: Unconstrained variance of Sav when observers are distinct
vsav2: Unconstrained variance of Sav when observers are homogeneous
p0sav1: Probability of overall agreement due to chance when observers are distinct
p0sav2: Probability of overall agreement due to chance when observers are homogeneous
resp[i,j]: Response for observer i on subject j; transpose of X (BEWARE)
score(i): Score associated with i'th category
call: As per sys.call(), to allow for using update

Arguments

X

A matrix or data-frame with observations/subjects as rows and observers as columns.

weights

"unweighted": For nominal categories - only perfect agreement is counted.

"linear"

For ordinal categories where disagreement is proportional to the distance between the categories. This is analogous to the agreement weights \(w_{i,j}=1-|score[i]-score[j]|/(max(score)-min(score))\).

"quadratic"

For ordinal categories where disagreement is proportional to the square of the distance between the categories. This is analogous to the agreement weights \(w_{i,j}=1-(score[i]-score[j])^2/(max(score)-min(score))^2\).

For nominal categories - only perfect agreement is counted.
For ordinal categories where disagreement is proportional to the distance between the categories. This is analogous to the agreement weights \(w_{i,j}=1-|score[i]-score[j]|/(max(score)-min(score))\).
For ordinal categories where disagreement is proportional to the square of the distance between the categories. This is analogous to the agreement weights \(w_{i,j}=1-(score[i]-score[j])^2/(max(score)-min(score))^2\).

This argument takes precedence over weights if it is specified.

score

The scores that are to be assigned to the categories. Currently, this defaults to 1:L, where L is the number of categories.

Details

The Fortran code from Professor Dianne O'Connell was adapted for R.

The output object is very similar to the Fortan code. Not all of the variance terms are currently used in the print, summary and plot methods.

Examples

Run this code


## Table 1 (O'Connell and Dobson, 1984)
summary(fit <- oconnell(landis, weights="unweighted"))
update(fit, weights="linear")
update(fit, weights="quadratic")

## Table 3 (O'Connell and Dobson, 1984)
slideTypeGroups <-
    list(c(2,3,5,26,31,34,42,58,59,67,70,81,103,120),
         c(7,10:13,17,23,30,41,51,55,56,60,65,71,73,76,86,87,105,111,116,119,124),
         c(4,6,24,25,27,29,39,48,68,77,79,94,101,102,117),
         c(9,32,36,44,52,62,84,95),
         c(35,53,69,72),
         c(8,15,18,19,47,64,82,93,98,99,107,110,112,115,121),
         c(1,16,22,49,63,66,78,90,100,113),
         c(28,37,40,61,108,114,118),
         106,
         43,
         83,
         c(54,57,88,91,126),
         c(74,104),
         38,
         46,
         c(89,122),
         c(80,92,96,123),
         85)
data.frame(SlideType=1:18,
           S1=sapply(slideTypeGroups,
                     function(ids) mean(fit$s1[as.character(ids)])),
           S2=sapply(slideTypeGroups,
                     function(ids) mean(fit$s2[as.character(ids)])))

## Table 5, O'Connell and Dobson (1984)
oconnell(landis==1)
oconnell(landis==2)
oconnell(landis==3)
oconnell(landis==4)
oconnell(landis==5)

## Plot of the marginal distributions
plot(fit)