Learn R Programming

IMP (version 1.1)

interPerfMeasures: Interactive Model Performance Evaluation & Comparison

Description

Interactive version of the staticPerfMeasures function

Usage

interPerfMeasures(list_models, sample_size_concord = 5000, model_function = NULL, data = NULL, y = NULL)

Arguments

list_models
A list of one (or more) dataframes for each model whose performance is to be evaluated. Each dataframe should comprise of 2 columns with the first column indicating the class labels (0 or 1) and the second column providing the raw predicted probabilities
sample_size_concord
For computing concordance-discordance measures (and c-statistic) a random sample is drawn from each dataset (if nrow(dataset) > 5000). Default sample size of 5000 can be adjusted by changing the value of this argument
model_function
Models can be created interactively, if required. For this option to work, a model function should be passed as an argument. The model function should take a formula as an argument, and return a a dataframe as output (dataframe should comprise of 2 columns with the first column indicating the class labels (0 or 1) and the second column provding the raw predicted probabilities) Refer to the example section for more details
data
The name of the data-set. The Independent Variable (IV) names, for interactive model building, is picked up from this data set
y
The column name of the Dependent Variable (DV), for interactive model building

Value

This function will launch a ShinyApp.Input parameters (such as the number of bins, the "g" argument in the static version of this function) can be adjusted through app widgets. The 'Run-Analysis' button in the app, will generate model performance output basis selected input parametersFor interactive Model building, a model function, data set & the dependent variable name should be passed as arguments. Interactive model building option creates additional input widgets in the app. This includes -A drop down to select independent variables (the names of the variables will be picked up from the data argument)An input slider to include additional models (upto 4 additional models can be created). Each additional model updates the original model created. For e.g. consider the dataset has 10 IVs: x1-x10. Original model was created by selecting x1-x4 from the drop down list. If we need to create a second model, by including x5 and excluding x3 simply type, "+ x5 - x3" in the input text box

Examples

Run this code
# Without interactive model development
model_1 <- glm(Species ~ Sepal.Length,data=iris,family=binomial)
model_2 <- glm(Species ~ Sepal.Width, data=iris, family = binomial)
df1 <- data.frame(model_1$y,fitted(model_1))
df2 <- data.frame(model_2$y,fitted(model_2))

## Not run: 
# #This will launch a Shiny App
# interPerfMeasures(list_models = list(df1,df2))## End(Not run)

# With interactive model development
glm_model <- function(formula) {
   glm_model <- glm(formula, data = iris, family = "binomial")
   out <- data.frame(glm_model$y, fitted(glm_model))
   out }
 ## Not run: 
#  #This will launch a Shiny App
#  interPerfMeasures (model_function = glm_model,data=iris,y="Species")## End(Not run)

Run the code above in your browser using DataLab