perf_eva: Binomial Metrics

Description

perf_eva calculates metrics to evaluate the performance of binomial classification model. It can also creates confusion matrix and model performance graphics.

Usage

perf_eva(pred, label, title = NULL, binomial_metric = c("mse", "rmse",
  "logloss", "r2", "ks", "auc", "gini"), confusion_matrix = FALSE,
  threshold = NULL, show_plot = c("ks", "lift"), pred_desc = TRUE,
  positive = "bad|1", ...)

Value

A list of binomial metric, confusion matrix and graphics

Arguments

pred: A list or vector of predicted probability or score.
label: A list or vector of label values.
title: The title of plot. Defaults to NULL.
binomial_metric: Defaults to c('mse', 'rmse', 'logloss', 'r2', 'ks', 'auc', 'gini'). If it is NULL, then no metric will calculated.
confusion_matrix: Logical, whether to create a confusion matrix. Defaults to TRUE.
threshold: Confusion matrix threshold. Defaults to the pred on maximum F1.
show_plot: Defaults to c('ks', 'roc'). Accepted values including c('ks', 'lift', 'gain', 'roc', 'lz', 'pr', 'f1', 'density').
pred_desc: whether to sort the argument of pred in descending order. Defaults to TRUE.
positive: Value of positive class. Defaults to "bad|1".
...: Additional parameters.

Details

Accuracy = true positive and true negative/total cases

Error rate = false positive and false negative/total cases

TPR, True Positive Rate(Recall or Sensitivity) = true positive/total actual positive

PPV, Positive Predicted Value(Precision) = true positive/total predicted positive

TNR, True Negative Rate(Specificity) = true negative/total actual negative = 1-FPR

NPV, Negative Predicted Value = true negative/total predicted negative

Examples

Run this code

# \donttest{
# load germancredit data
data("germancredit")
# filter variable via missing rate, iv, identical value rate
dtvf = var_filter(germancredit, "creditability")

# breaking dt into train and test
dt_list = split_df(dtvf, "creditability")
label_list = lapply(dt_list, function(x) x$creditability)

# woe binning
bins = woebin(dt_list$train, "creditability")
# scorecard, prob
cardprob = scorecard2(bins, dt = dt_list, y = 'creditability', return_prob = TRUE)

# credit score
score_list = lapply(dt_list, function(x) scorecard_ply(x, cardprob$card))

###### perf_eva examples ######
# Example I, one datset
## predicted p1
perf_eva(pred = cardprob$prob$train, label=label_list$train,
         title = 'train')
## predicted score
# perf_eva(pred = score_list$train, label=label_list$train,
#   title = 'train')

# Example II, multiple datsets
## predicted p1
perf_eva(pred = cardprob$prob, label = label_list,
         show_plot = c('ks', 'lift', 'gain', 'roc', 'lz', 'pr', 'f1', 'density'))
## predicted score
# perf_eva(score_list, label_list)

# }