MachineShop (version 3.3.0)

resample: Resample Estimation of Model Performance

Description

Estimation of the predictive performance of a model estimated and evaluated on training and test samples generated from an observed data set.

Usage

resample(...)
# S3 method for default
resample(object, model = NULL, control = MachineShop::settings("control"), ...)
# S3 method for formula
resample(formula, data, model, ...)
# S3 method for matrix
resample(x, y, model, ...)
# S3 method for ModelFrame
resample(input, model = NULL, ...)
# S3 method for recipe
resample(input, model = NULL, ...)
# S3 method for ModelSpecification
resample(object, ...)
# S3 method for MLModel
resample(model, ...)
# S3 method for MLModelFunction
resample(model, ...)

Arguments

...

arguments passed from the generic function to its methods and from the MLModel and MLModelFunction methods to others. The first argument of each resample method is positional and, as such, must be given first in calls to them.

object

model input or specification.

model

model function, function name, or object; or another object that can be coerced to a model. A model can be given first followed by any of the variable specifications, and the argument can be omitted altogether in the case of modeled inputs.

control

control function, function name, or object defining the resampling method to be employed.

formula, data

formula defining the model predictor and response variables and a data frame containing them.

x, y

matrix and object containing predictor and response variables.

input

input object defining and containing the model predictor and response variables.

Value

Resample class object.

Details

Stratified resampling is performed automatically for the formula and matrix methods according to the type of response variable. In general, strata are constructed from numeric proportions for BinomialVariate; original values for character, factor, logical, and ordered; first columns of values for matrix; original values for numeric; and numeric times within event statuses for Surv. Numeric values are stratified into quantile bins and categorical values into factor levels defined by MLControl.

Resampling stratification variables may be specified manually for ModelFrames upon creation with the strata argument in their constructor. Resampling of this class is unstratified by default.

Stratification variables may be designated in recipe specifications with the role_case function. Resampling will be unstratified otherwise.

See Also

c, metrics, performance, plot, summary

Examples

# NOT RUN {
## Requires prior installation of suggested package gbm to run

## Factor response example

fo <- Species ~ .
control <- CVControl()

gbm_res1 <- resample(fo, iris, GBMModel(n.trees = 25), control)
gbm_res2 <- resample(fo, iris, GBMModel(n.trees = 50), control)
gbm_res3 <- resample(fo, iris, GBMModel(n.trees = 100), control)

summary(gbm_res1)
plot(gbm_res1)

res <- c(GBM1 = gbm_res1, GBM2 = gbm_res2, GBM3 = gbm_res3)
summary(res)
plot(res)
# }
# NOT RUN {
# }