Learn R Programming

MachineShop (version 3.5.0)

metrics: Performance Metrics

Description

Compute measures of agreement between observed and predicted responses.

Usage

accuracy(
  observed,
  predicted = NULL,
  weights = NULL,
  cutoff = MachineShop::settings("cutoff"),
  ...
)

auc( observed, predicted = NULL, weights = NULL, multiclass = c("pairs", "all"), metrics = c(MachineShop::tpr, MachineShop::fpr), stat = MachineShop::settings("stat.Curve"), ... )

brier(observed, predicted = NULL, weights = NULL, ...)

cindex(observed, predicted = NULL, weights = NULL, ...)

cross_entropy(observed, predicted = NULL, weights = NULL, ...)

f_score( observed, predicted = NULL, weights = NULL, cutoff = MachineShop::settings("cutoff"), beta = 1, ... )

fnr( observed, predicted = NULL, weights = NULL, cutoff = MachineShop::settings("cutoff"), ... )

fpr( observed, predicted = NULL, weights = NULL, cutoff = MachineShop::settings("cutoff"), ... )

kappa2( observed, predicted = NULL, weights = NULL, cutoff = MachineShop::settings("cutoff"), ... )

npv( observed, predicted = NULL, weights = NULL, cutoff = MachineShop::settings("cutoff"), ... )

ppr( observed, predicted = NULL, weights = NULL, cutoff = MachineShop::settings("cutoff"), ... )

ppv( observed, predicted = NULL, weights = NULL, cutoff = MachineShop::settings("cutoff"), ... )

pr_auc( observed, predicted = NULL, weights = NULL, multiclass = c("pairs", "all"), ... )

precision( observed, predicted = NULL, weights = NULL, cutoff = MachineShop::settings("cutoff"), ... )

recall( observed, predicted = NULL, weights = NULL, cutoff = MachineShop::settings("cutoff"), ... )

roc_auc( observed, predicted = NULL, weights = NULL, multiclass = c("pairs", "all"), ... )

roc_index( observed, predicted = NULL, weights = NULL, cutoff = MachineShop::settings("cutoff"), fun = function(sensitivity, specificity) (sensitivity + specificity)/2, ... )

sensitivity( observed, predicted = NULL, weights = NULL, cutoff = MachineShop::settings("cutoff"), ... )

specificity( observed, predicted = NULL, weights = NULL, cutoff = MachineShop::settings("cutoff"), ... )

tnr( observed, predicted = NULL, weights = NULL, cutoff = MachineShop::settings("cutoff"), ... )

tpr( observed, predicted = NULL, weights = NULL, cutoff = MachineShop::settings("cutoff"), ... )

weighted_kappa2(observed, predicted = NULL, weights = NULL, power = 1, ...)

gini(observed, predicted = NULL, weights = NULL, ...)

mae(observed, predicted = NULL, weights = NULL, ...)

mse(observed, predicted = NULL, weights = NULL, ...)

msle(observed, predicted = NULL, weights = NULL, ...)

r2( observed, predicted = NULL, weights = NULL, method = c("mse", "pearson", "spearman"), distr = character(), ... )

rmse(observed, predicted = NULL, weights = NULL, ...)

rmsle(observed, predicted = NULL, weights = NULL, ...)

Arguments

observed

observed responses; or confusion, performance curve, or resample result containing observed and predicted responses.

predicted

predicted responses if not contained in observed.

weights

numeric vector of non-negative case weights for the observed responses [default: equal weights].

cutoff

numeric (0, 1) threshold above which binary factor probabilities are classified as events and below which survival probabilities are classified. If NULL, then confusion matrix-based metrics are computed on predicted class probabilities if given.

...

arguments passed to or from other methods.

multiclass

character string specifying the method for computing generalized area under the performance curve for multiclass factor responses. Options are to average over areas for each pair of classes ("pairs") or for each class versus all others ("all").

metrics

vector of two metric functions or function names that define a curve under which to calculate area [default: ROC metrics].

stat

function or character string naming a function to compute a summary statistic at each cutoff value of resampled metrics in performance curves, or NULL for resample-specific metrics.

beta

relative importance of recall to precision in the calculation of f_score [default: F1 score].

fun

function to calculate a desired sensitivity-specificity tradeoff.

power

power to which positional distances of off-diagonals from the main diagonal in confusion matrices are raised to calculate weighted_kappa2.

method

character string specifying whether to compute r2 as the coefficient of determination ("mse") or as the square of "pearson" or "spearman" correlation.

distr

character string specifying a distribution with which to estimate the observed survival mean in the total sum of square component of r2. Possible values are "empirical" for the Kaplan-Meier estimator, "exponential", "extreme", "gaussian", "loggaussian", "logistic", "loglogistic", "lognormal", "rayleigh", "t", or "weibull". Defaults to the distribution that was used in predicting mean survival times.

References

Hand, D. J., & Till, R. J. (2001). A simple generalisation of the area under the ROC curve for multiple class classification problems. Machine Learning, 45, 171-186.

See Also

metricinfo, performance