evaluate: Evaluate the accuracy of multiple (candidate) classifiers in several subgroups

Description

Assess classification accuracy of multiple classifcation rules stratified by subgroups, e.g. in diseased (sensitivity) and healthy (specificity) individuals.

Usage

evaluate(
  data,
  contrast = define_contrast("raw"),
  benchmark = 0.5,
  alpha = 0.05,
  alternative = c("two.sided", "greater", "less"),
  adjustment = c("none", "bonferroni", "maxt", "bootstrap", "mbeta"),
  transformation = c("none", "logit", "arcsin"),
  analysis = c("co-primary", "full"),
  regu = FALSE,
  pars = list(),
  ...
)

Value

(cases_results)

list of analysis results including (adjusted) confidence intervals and p-values

Arguments

data: (list)
of n_g x m binary matrix or data.frame (n_g observations of m binary decisions), g is the index of subgroups/classes, usually created via compare.
contrast: (cases_contrast)
specified via define_contrast
benchmark: (numeric)
value to compare against (RHS), should have same length as data.
alpha: (numeric)
significance level (default: 0.05)
alternative: (character)
specification of alternative hypothesis
adjustment: (character)
specification of statistical adjustment taken to address multiplicity. The default 'none' does not perform any adjustment for multiplicity.
transformation: (character)
define transformation to ensure results (e.g. point estimates, confidence limits) lie in unit interval ("none" (default), "logit", or "arcsin" (sqrt))
analysis: (character)
"co-primary" or "full"
regu: (numeric | logical)
vector of length 3, specify type of shrinkage. Alternatively, logical of length one (TRUE := c(1, 1/2, 1/4), FALSE := c(0, 0, 0))
pars: (list)
further parameters given as named list list(type="pairs", nboot=2000)
...: (any)
additional named parameters, can be used instead of (in in conjunction with) pars

Details

Adjustment methods (adjustment) and additional parameters (pars or ...):

"none" (default): no adjustment for multiplicity

"bonferroni": Bonferroni adjustment

"maxt": maxT adjustment, based on a multivariate normal approximation of the vector of test statistics

"bootstrap": Bootstrap approach

nboot: number of bootstrap draws (default: 2000)
type: type of bootstrap, "pairs" (default) or "wild"
dist: residual distribution for wild bootstrap, "Normal" (default) or "Rademacher"
proj_est: should bootstrapped estimates for wild bootstrap be projected into unit interval? (default: TRUE)
res_tra: type of residual transformation for wild boostrap, 0,1,2 or 3 (default: 0 = no transformation) (for details on res_tra options, see this presentation by James G. MacKinnon (2012) and references therein)

"mbeta": A heuristic Bayesian approach which is based on a multivariate beta-binomial model.

nrep: number of posterior draws (default: 5000)
lfc_pr: prior probability of 'least-favorable parameter configuration' (default: 1 if analysis == "co-primary", 0 if analysis == "full").

Examples

Run this code

#
data <- draw_data_roc()
evaluate(data)

Run the code above in your browser using DataLab