xxIRT (version 2.0.1)

cat_sim: Computerized Adaptive Testing (CAT) Simulation

Description

cat_sim simulates CAT sessions with user-defined algorithms

cat_select_default selects the most informative item

cat_select_ccat selects items under content-balancing constraint (see Kingsbury & Zara, 1989, 1991)

cat_select_shadow implements the shadow test algorithm described in van der Linden (2010)

cat_stop_projection implements the projection-based stopping rule described in Luo et al (2016)

Usage

cat_sim(theta.true, pool, opts, cat.select = cat_select_default,
  cat.estimate = cat_estimate_default, cat.stop = cat_stop_default,
  debug = FALSE)
# S3 method for cat
print(x, ...)
# S3 method for cat
plot(x, ...)
cat_select_default(cat.data)
cat_select_ccat(cat.data)
cat_select_shadow(cat.data)
cat_estimate_default(cat.data)
cat_stop_default(cat.data)
cat_stop_projection(cat.data)

Arguments

theta.true

the true theta parameter

pool

a data frame of items used as the item pool

opts

a list of option parameters (min and max are required)

cat.select

the item selection rule

cat.estimate

the theta estimation rule

cat.stop

the stopping rule

debug

TRUE to turn on the debugging mode

x

a cat object

...

further arguments

cat.data

a list of CAT data

Value

cat_sim returns a cat object (see details section)

cat_estimate_default estimates the ability using EAP (all correct/incorrect responses) or MLE (mixed responses)

cat_stop_default evalutes whether to stop the CAT using the SE rule, MI rule, or CI rule

Details

All data (input and output) are combined into a list named cat.data, which includes options (opts), true theta (true), estiamted theta (est), item pool (pool), administered items (items), statistics (stats), administration history (admin), test length (len), and debugging switch (debug).

To write a new cat.select function, make sure it only takes cat.data as input and outputs a list with the selected item, updated pool, and output for additional output (optional). Retrieve the additional output using output.select from the returnig cat object.

To write a new cat.estimate function, make sure it only takes cat.data as input and outputs a list with the estimated theta, and output for additional output (optional). Retrieve the additional output using output.estimate from the returnig cat object.

To write a new cat.stop function, make sure it only takes cat.data as input and outputs a list with the boolean stop decision, and output for additional output (optional). Retrieve the additional output using output.stop from the returnig cat object.

cat_select_default selects the most informative item for current theta. When randomesque is set, it randomly selects an item from the k most informative ones to control item exposure rate.

To use cat_select_ccat, set target (percentage) using ccat_target and initial randomness using ccat_random in options.

To use cat_select_shadow, pass in a data frame of constraints with 4 columns to opts$shadow.constraints: name, level, min, and max.

The cat_stop_default evaluates one of the three criteria after reaching minimum lenght: (1) if opts$stop.se is set, evalute if the se reaches the threshold; (2) if opts$stop.mi is set, evalute if all item reach the threshold; (3) if opts$stop.cut is set, evalute if the 95

To use cat_stop_projection, pass in a data frame of constraints with 4 columns to opts$projection.constraints: name, level, min, and max. Also set a method in opts$method ('information' or 'difficulty') and a cut score in opts$cut.

Examples

# NOT RUN {
### generate a 200-item pool
pool <- irt_model("3pl")$gendata(1,200)$items
pool$content <- sample(1:3, nrow(pool), replace=TRUE)
pool$time <- round(exp(rnorm(nrow(pool), log(60), .2)))

### ex. 1: 10-30 items
### maximum information selection rule
### standard error stopping rule (se=.3)
opts <- list(min=10, max=30, stop.se=.3)
x <- cat_sim(0.1, pool, opts)
x$admin
plot(x)

### ex. 2: 10-30 items
### maximum information selection rule
### minimum information stopping rule (mi=.3)
opts <- list(min=10, max=30, stop.mi=.8)
x <- cat_sim(0.1, pool, opts)
x$admin
plot(x)

### ex. 3: 10-30 items
### maximum information selection rule
### confidence interval stopping rule (cut=0)
opts <- list(min=10, max=30, stop.cut=0)
x <- cat_sim(0.1, pool, opts)
x$admin
plot(x)

### ex. 4: 10-30 items
### maximum information selection rules, randomesque = 5
### standard error stopping rule (se=.3) 
opts <- list(min=10, max=30, stop.se=.3, randomesque=5)
x <- cat_sim(0.1, pool, opts)
x$admin
plot(x)

### ex. 5: 10-30 items
### c-cat selection rule, first 10 areas are random
### confidence interval stopping rule
opts <- list(min=30, max=60, stop.cut=0, 
    ccat.target=c(.50,.25,.25), ccat.random=10)
x <- cat_sim(0.1, pool, opts, cat.select=cat_select_ccat)
x$admin
plot(x)
freq(x$admin$content, 1:3)

# }
# NOT RUN {
### ex. 6: 15 items
### shadow test selection rule
### content: [5, 5, 5] items in area 1--3
### response time: avg. 55--65 seconds
cons <- data.frame(name="content", level=c(1,2,3), min=c(5, 5, 5), 
                   max=c(5, 5, 5), stringsAsFactors=FALSE)
cons <- rbind(cons, c("time", NA, 55*15, 65*15))
opts <- list(min=15, max=15, stop.se=.03, shadow.constraints=cons)
x <- cat_sim(0.1, pool, opts, cat.select=cat_select_shadow)
x$admin
plot(x)
freq(x$admin$content, 1:3)
mean(x$items$time)

### ex. 7: 20-30 items
### shadow selection rule
### projection-based stopping rule
cons <- data.frame(name="content", level=c(1,2,3), min=c(10, 10, 10), 
                   max=c(10, 10, 10), stringsAsFactors=FALSE)
cons <- rbind(cons, c("time", NA, 55*30, 65*30))
opts <- list(min=20, max=30, projection.cut=0, projection.constraints=cons, 
projection.method="difficulty", shadow.constraints=cons)
x <- cat_sim(0.1, pool, opts, cat.select=cat_select_shadow, 
cat.stop=cat_stop_projection)
x$admin
plot(x)
# }