IMP (version 1.1)

interPerfMeasures: Interactive Model Performance Evaluation & Comparison

Description

Interactive version of the staticPerfMeasures function

Usage

interPerfMeasures(list_models, sample_size_concord = 5000, model_function = NULL, data = NULL, y = NULL)

Arguments

list_models

A list of one (or more) dataframes for each model whose performance is to be evaluated. Each dataframe should comprise of 2 columns with the first column indicating the class labels (0 or 1) and the second column providing the raw predicted probabilities

sample_size_concord

For computing concordance-discordance measures (and c-statistic) a random sample is drawn from each dataset (if nrow(dataset) > 5000). Default sample size of 5000 can be adjusted by changing the value of this argument

model_function

Models can be created interactively, if required. For this option to work, a model function should be passed as an argument. The model function should take a formula as an argument, and return a a dataframe as output (dataframe should comprise of 2 columns with the first column indicating the class labels (0 or 1) and the second column provding the raw predicted probabilities) Refer to the example section for more details

data

The name of the data-set. The Independent Variable (IV) names, for interactive model building, is picked up from this data set

y

The column name of the Dependent Variable (DV), for interactive model building

Value

This function will launch a ShinyApp.Input parameters (such as the number of bins, the "g" argument in the static version of this function) can be adjusted through app widgets. The 'Run-Analysis' button in the app, will generate model performance output basis selected input parametersFor interactive Model building, a model function, data set & the dependent variable name should be passed as arguments. Interactive model building option creates additional input widgets in the app. This includes -A drop down to select independent variables (the names of the variables will be picked up from the data argument)An input slider to include additional models (upto 4 additional models can be created). Each additional model updates the original model created. For e.g. consider the dataset has 10 IVs: x1-x10. Original model was created by selecting x1-x4 from the drop down list. If we need to create a second model, by including x5 and excluding x3 simply type, "+ x5 - x3" in the input text box

Examples

# Without interactive model development
model_1 <- glm(Species ~ Sepal.Length,data=iris,family=binomial)
model_2 <- glm(Species ~ Sepal.Width, data=iris, family = binomial)
df1 <- data.frame(model_1$y,fitted(model_1))
df2 <- data.frame(model_2$y,fitted(model_2))

## Not run: 
# #This will launch a Shiny App
# interPerfMeasures(list_models = list(df1,df2))## End(Not run)

# With interactive model development
glm_model <- function(formula) {
   glm_model <- glm(formula, data = iris, family = "binomial")
   out <- data.frame(glm_model$y, fitted(glm_model))
   out }
 ## Not run: 
#  #This will launch a Shiny App
#  interPerfMeasures (model_function = glm_model,data=iris,y="Species")## End(Not run)