sigr (version 1.1.5)

resampleScoreModelPair: Studentized bootstrap test of strength of scoreFn(yValues,model1Values) > scoreFn(yValues,model1Values).

Description

Studentized bootstrap test of strength of scoreFn(yValues,model1Values) > scoreFn(yValues,model1Values) sampled with replacement.

Usage

resampleScoreModelPair(
  model1Values,
  model2Values,
  yValues,
  scoreFn,
  ...,
  na.rm = FALSE,
  returnScores = FALSE,
  nRep = 100,
  parallelCluster = NULL,
  sameSample = FALSE
)

Value

summaries

Arguments

model1Values

numeric array of predictions (model to test).

model2Values

numeric array of predictions (reference model).

yValues

numeric/logical array of outcomes, dependent, or truth values

scoreFn

function with signature scoreFn(modelValues,yValues) returning scalar numeric score.

...

not used, forces later arguments to be bound by name.

na.rm

logical, if TRUE remove NA values

returnScores

logical if TRUE return detailed resampledScores.

nRep

integer number of repititions to perform.

parallelCluster

optional snow-style parallel cluster.

sameSample

logical if TRUE use the same sample in computing both scores during bootstrap replication (else use independent samples).

Details

True confidence intervals are harder to get right (see "An Introduction to the Bootstrap", Bradely Efron, and Robert J. Tibshirani, Chapman & Hall/CRC, 1993.), but we will settle for simple p-value estimates.

Examples

set.seed(25325)
y <- 1:5
m1 <- c(1,1,2,2,2)
m2 <- c(1,1,1,1,2)
cor(m1,y)
cor(m2,y)
f <- function(modelValues,yValues) {
  if((sd(modelValues)<=0)||(sd(yValues)<=0)) {
    return(0)
  }
  cor(modelValues,yValues)
}
resampleScoreModelPair(m1,m2,y,f)