mmpf (version 0.0.5)

permutationImportance: computes permutation importance

Description

computes the change in prediction error from permuting variables.

Usage

permutationImportance(data, vars, y, model, nperm = 100L,
  predict.fun = function(object, newdata) predict(object, newdata =
  newdata), loss.fun = function(x, y) defaultLoss(x, y),
  contrast.fun = function(x, y) x - y)

Arguments

data

a data.frame including both y and vars.

vars

a character vector specifying columns of data to permute.

y

a character vector giving the name of the target/outcome variable.

model

an object with a predict method which returns a vector or matrix. presumably this object represents a model fit.

nperm

positive integer giving the number of times to permute the indicated variables (default is 100).

predict.fun

what function to generate predictions using model. default is the predict method for model. the function must take two arguments, object and newdata and should return a vector or matrix.

loss.fun

what loss function to use to measure prediction errors. default is mean squared-error for ordered predictions and mean misclassification error for unordered prediction errors. this function must take two arguments, “x” and “y”, which operate on the output of predict.fun and data[, y].

contrast.fun

what function to use to contrast the permuted and unpermuted predictions. default is the difference. this function takes two arguments “x” and “y”, which are the output of the loss.fun.

Value

a numeric vector or matrix, depending on contrast.fun and loss.fun, giving the change in prediction error from nperm permutations of vars.

Examples

# NOT RUN {
X = replicate(3, rnorm(100))
y = X %*% runif(3)
data = data.frame(X, y)
fit = lm(y ~ -1 + X1 + X2 + X3, data)

permutationImportance(data, "X1", "y", fit)
# }