Helpers to be used alongside metric_vec_template() and
metric_summarizer() when creating new metrics. See Custom performance metrics for more
information.
get_weights(data, estimator)finalize_estimator(x, estimator = NULL, metric_class = "default")
finalize_estimator_internal(metric_dispatcher, x, estimator)
dots_to_estimate(data, ...)
validate_estimator(estimator, estimator_override = NULL)
A table with truth values as columns and predicted values as rows.
Either NULL for auto-selection, or a single character
for the type of estimator to use.
The column used to autoselect the estimator. This is generally
the truth column, but can also be a table if your metric has table methods.
A single character of the name of the metric to autoselect
the estimator for. This should match the method name created for
finalize_estimator_internal().
A simple dummy object with the class provided to
metric_class. This is created and passed along for you.
A set of unquoted column names or one or more
dplyr selector functions to choose which variables contain the
class probabilities. If truth is binary, only 1 column should be selected.
Otherwise, there should be as many columns as factor levels of truth.
A character vector overriding the default allowed
estimator list of
c("binary", "macro", "micro", "macro_weighted"). Set
this if your classification estimator does not support all of these methods.
get_weights() accepts a confusion matrix and an estimator of type
"macro", "micro", or "macro_weighted" and returns the correct weights.
It is useful when creating multiclass metrics.
finalize_estimator() is the engine for auto-selection of estimator based
on the type of x. Generally x is the truth column. This function
is called from the vector method of your metric.
finalize_estimator_internal() is an S3 generic that you should extend for
your metric if it does not implement only the following estimator types:
"binary", "macro", "micro", and "macro_weighted".
If your metric does support all of these, the default version of
finalize_estimator_internal() will autoselect estimator appropriately.
If you need to create a method, it should take the form:
finalize_estimator_internal.metric_name. Your method for
finalize_estimator_internal() should do two things:
If estimator is NULL, autoselect the estimator based on the
type of x and return a single character for the estimator.
If estimator is not NULL, validate that it is an allowed estimator
for your metric and return it.
If you are using the default for finalize_estimator_internal(), the
estimator is selected using the following heuristics:
If estimator is not NULL, it is validated and returned immediately
as no auto-selection is needed.
If x is a:
factor - Then "binary" is returned if it has 2 levels, otherwise
"macro" is returned.
numeric - Then "binary" is returned.
table - Then "binary" is returned if it has 2 columns, otherwise
"macro" is returned. This is useful if you have table methods.
matrix - Then "macro" is returned.
dots_to_estimate() is useful with class probability metrics that take
... rather than estimate as an argument. It constructs either a single
name if 1 input is provided to ... or it constructs a quosure where the
expression constructs a matrix of as many columns as are provided to ....
These are eventually evaluated in the summarise() call in
metric_summarizer() and evaluate to either a vector or a matrix for further
use in the underlying vector functions.
validate_estimator() is called from your metric specific method of
finalize_estimator_internal() and ensures that a user provided estimator
is of the right format and is one of the allowed values.
metric_summarizer() metric_vec_template()