Some example rating datasets for the immer package.
data(data.immer01a)
data(data.immer01b)
data(data.immer02)
data(data.immer03)
data(data.immer04a)
data(data.immer04b)
data(data.immer05)
data(data.immer06)
data(data.immer07)
data(data.immer08)
data(data.immer09)
data(data.immer10)
data(data.immer11)
data(data.immer12)The format of the dataset data.immer01a is:
'data.frame': 23904 obs. of 8 variables:
$ idstud: int 10001 10001 10003 10003 10003 10004 10004 10005 10005 10006 ...
$ type : Factor w/ 2 levels "E","I": 1 2 1 1 2 1 2 1 2 1 ...
$ rater : Factor w/ 57 levels "R101","R102",..: 1 36 33 20 21 57 36 9 31 21 ...
$ k1 : int 2 1 0 0 0 2 2 1 2 0 ...
$ k2 : int 1 1 0 0 0 1 1 1 2 0 ...
$ k3 : int 1 1 0 0 0 1 1 1 2 1 ...
$ k4 : int 2 2 1 0 0 1 1 1 2 1 ...
$ k5 : int 1 2 0 0 0 2 1 2 3 2 ...
The format of the dataset data.immer01b is:
'data.frame': 4244 obs. of 8 variables:
$ idstud: int 10001 10003 10005 10007 10009 10016 10018 10022 10024 10029 ...
$ type : Factor w/ 1 level "E": 1 1 1 1 1 1 1 1 1 1 ...
$ rater : Factor w/ 20 levels "R101","R102",..: 1 20 9 5 14 19 20 6 10 10 ...
$ k1 : int 2 0 1 2 2 2 3 1 3 2 ...
$ k2 : int 1 0 1 2 2 1 3 2 2 1 ...
$ k3 : int 1 0 1 1 3 2 2 1 3 1 ...
$ k4 : int 2 0 1 2 3 2 2 2 3 2 ...
$ k5 : int 1 0 2 1 3 1 2 3 3 1 ...
This dataset is a subset of data.immer01a.
The format of the dataset data.immer02 is:
'data.frame': 6105 obs. of 6 variables:
$ idstud: int 10002 10004 10005 10006 10007 10008 10009 10010 10013 10014 ...
$ rater : Factor w/ 44 levels "DR101","DR102",..: 43 15 12 21 9 3 35 24 11 17 ...
$ a1 : int 3 1 2 1 0 2 1 2 1 1 ...
$ a2 : int 3 0 3 1 0 3 0 2 2 1 ...
$ a3 : int 1 2 0 1 2 3 2 2 1 1 ...
$ a4 : int 2 1 2 1 1 3 1 2 2 1 ...
The format of the dataset data.immer03 is:
'data.frame': 6466 obs. of 6 variables:
$ idstud: int 10001 10002 10003 10004 10005 10006 10007 10009 10010 10012 ...
$ rater : Factor w/ 44 levels "R101","R102",..: 18 10 8 25 19 31 16 22 29 6 ...
$ b1 : int 1 2 1 3 3 2 3 2 2 1 ...
$ b2 : int 2 1 0 3 3 1 1 2 2 1 ...
$ b3 : int 2 3 1 2 3 1 2 2 2 2 ...
$ b4 : int 1 2 0 2 2 2 3 2 3 1 ...
The format of the dataset data.immer04a is:
'data.frame': 25578 obs. of 7 variables:
$ idstud: int 10001 10001 10001 10002 10002 10002 10003 10003 10004 10004 ...
$ task : Factor w/ 4 levels "l1","l2","s1",..: 1 4 4 1 1 3 1 3 2 2 ...
$ rater : Factor w/ 43 levels "R101","R102",..: 14 31 25 39 35 19 43 27 12 4 ...
$ TA : int 5 2 4 0 0 0 2 6 5 3 ...
$ CC : int 4 1 3 1 0 0 2 6 4 3 ...
$ GR : int 4 1 2 1 0 0 1 7 5 2 ...
$ VOC : int 4 2 3 1 0 0 1 6 5 3 ...
The format of the dataset data.immer04b is:
'data.frame': 2975 obs. of 7 variables:
$ idstud: int 10002 10004 10010 10013 10015 10016 10024 10025 10027 10033 ...
$ task : Factor w/ 1 level "s1": 1 1 1 1 1 1 1 1 1 1 ...
$ rater : Factor w/ 20 levels "R101","R102",..: 19 1 5 16 13 13 8 10 19 5 ...
$ TA : int 0 3 5 5 3 2 3 6 4 5 ...
$ CC : int 0 3 4 5 4 1 4 7 3 3 ...
$ GR : int 0 3 3 6 5 2 3 6 3 2 ...
$ VOC : int 0 2 4 6 5 2 3 6 3 2 ...
This dataset is a subset of data.immer04a.
The format of the dataset data.immer05 is:
'data.frame': 21398 obs. of 9 variables:
$ idstud : int 10001 10001 10002 10002 10003 10003 10004 10004 10005 10005 ...
$ type : Factor w/ 2 levels "l","s": 2 1 2 1 2 1 2 1 2 1 ...
$ task : Factor w/ 6 levels "l1","l4","l5",..: 5 2 6 3 5 1 5 1 5 2 ...
$ rater : Factor w/ 41 levels "ER101","ER102",..: 1 40 38 23 37 33 2 33 21 27 ...
$ idstud_task: Factor w/ 19484 levels "10001l4","10001s3",..: 2 1 4 3 6 5 8 7 10 9 ...
$ TA : int 3 4 6 6 4 2 0 3 1 3 ...
$ CC : int 5 4 5 5 3 3 0 2 5 3 ...
$ GR : int 4 4 5 6 5 3 0 4 5 4 ...
$ VO : int 6 4 6 6 4 3 0 3 4 3 ...
The dataset data.immer06 is a string containing
an input syntax for the FACETS program.
The format of the dataset data.immer07 is:
'data.frame': 1500 obs. of 6 variables:
$ pid : int 1 1 1 2 2 2 3 3 3 4 ...
$ rater: chr "R1" "R2" "R3" "R1" ...
$ I1 : num 1 1 2 1 1 1 0 1 1 2 ...
$ I2 : num 0 1 1 2 1 2 1 1 2 1 ...
$ I3 : num 1 1 2 0 0 1 1 0 2 1 ...
$ I4 : num 0 0 1 0 0 1 0 1 2 0 ...
The format of the dataset data.immer08 (example in
Schuster & Smith, 2006) is
'data.frame': 16 obs. of 3 variables:
$ Facility: int 1 1 1 1 2 2 2 2 3 3 ...
$ Research: int 1 2 3 4 1 2 3 4 1 2 ...
$ weights : int 40 6 4 15 4 25 1 5 4 2 ...
The dataset data.immer09 contains reviewer ratings for
conference papers (Kuhlisch et al., 2016):
'data.frame': 128 obs. of 3 variables:
$ idpaper : int 1 1 1 2 2 3 3 3 4 4 ...
$ idreviewer: int 11 15 20 1 10 11 15 20 13 16 ...
$ score : num 7 7 7 7 7 7 7 7 7 7 ...
The dataset data.immer10 contains standard setting
ratings of 13 raters on 61 items (including item identifier item
and item difficulty itemdiff)
'data.frame': 61 obs. of 15 variables:
$ item : chr "I01" "I02" "I03" "I04" ...
$ itemdiff: num 380 388 397 400 416 425 427 434 446 459 ...
$ R01 : int 1 3 2 2 1 3 2 2 3 1 ...
$ R02 : int 1 1 1 1 1 2 1 2 2 1 ...
$ R03 : int 1 1 1 1 1 1 2 2 3 1 ...
$ R04 : int 1 2 1 3 2 2 2 2 3 2 ...
$ R05 : int 1 1 2 1 1 1 2 2 3 2 ...
$ R06 : int 1 2 1 1 1 2 2 2 3 2 ...
$ R07 : int 1 2 1 2 1 1 2 1 3 1 ...
$ R08 : int 2 2 1 2 1 1 2 2 3 2 ...
$ R09 : int 2 1 1 2 1 2 1 2 3 1 ...
$ R10 : int 2 2 2 2 1 2 2 3 3 2 ...
$ R11 : int 2 2 1 2 1 2 2 2 3 2 ...
$ R12 : int 2 2 1 3 1 2 2 2 3 2 ...
$ R13 : int 1 1 1 1 1 1 1 1 2 1 ...
The dataset data.immer11 contains ratings of 148 cases (screening
mammogram samples) diagnoses by 110 raters (Zhang & Petersen, xxxx). The codes of the
polytomous rating are normal (code 0), benign (code 1), probably benign (code 2),
possibly malignant (code 3), and probably malignant (code 4). The dataset was extracted
from an image plot in Figure 2 by using the processing function
png::readPNG. The format of the dataset is
'data.frame': 148 obs. of 110 variables:
$ R001: num 2 1 3 2 1 2 0 0 0 2 ...
$ R002: num 1 3 4 4 0 4 0 0 3 0 ...
$ R003: num 0 0 0 4 0 2 3 0 0 0 ...
$ R004: num 1 2 1 4 2 2 2 0 4 4 ...
[... ]
The dataset data.immer12 contains ratings of the 2002 olympic pairs
figure skating competition. This dataset has been used in Lincare (2009). The items
are ST (short program, technical merit),
SA (short program, artistic impression),
FT (free program, technical merit), and
FA (free program, artistic impression). The format of the dataset is
'data.frame': 180 obs. of 7 variables:
$ idpair: int 1 1 1 1 1 1 1 1 1 2 ...
$ pair : chr "BB-Svk" "BB-Svk" "BB-Svk" "BB-Svk" ...
$ judge : chr "RUS" "CHI" "USA" "FRA" ...
$ ST : int 58 57 57 56 55 55 50 51 51 47 ...
$ SA : int 58 57 57 56 55 55 50 51 51 47 ...
$ FT : int 58 57 57 56 55 55 50 51 51 47 ...
$ FA : int 58 57 57 56 55 55 50 51 51 47 ...
Kuhlisch, W., Roos, M., Rothe, J., Rudolph, J., Scheuermann, B., & Stoyan, D. (2016). A statistical approach to calibrating the scores of biased reviewers of scientific papers. Metrika, 79, 37-57.
Linacre, J. M. (2009). Local independence and residual covariance: A study of Olympic figure skating ratings. Journal of Applied Measurement, 10(2), 157-169.
Schuster, C., & Smith, D. A. (2006). Estimating with a latent class model the reliability of nominal judgments upon which two raters agree. Educational and Psychological Measurement, 66(5), 739-747.
Zhang, S., & Petersen, J. H. (XXXX). Quantifying rater variation for ordinal data using a rating scale model. Statistics in Medicine, XX(xx), xxx-xxx.