mlr3measures (version 0.3.0)

bbrier: Binary Brier Score

Description

Brier score for binary classification problems defined as $$ \frac{1}{n} \sum_{i=1}^n (I_i - p_i)^2. $$ \(I_{i}\) is 1 if observation \(i\) belongs to the positive class, and 0 otherwise.

Note that this (more common) definition of the Brier score is equivalent to the original definition of the multi-class Brier score (see mbrier()) divided by 2.

Usage

bbrier(truth, prob, positive, ...)

Arguments

truth

:: factor() True (observed) labels. Must have the exactly same two levels and the same length as response.

prob

:: numeric() Predicted probability for positive class. Must have exactly same length as truth.

positive

:: character(1) Name of the positive class.

...

:: any Additional arguments. Currently ignored.

Value

Performance value as numeric(1).

Meta Information

• Type: "binary"

• Range: \([0, 1]\)

• Minimize: TRUE

• Required prediction: prob

References

https://en.wikipedia.org/wiki/Brier_score

Brier GW (1950). “Verification of forecasts expressed in terms of probability.” Monthly Weather Review, 78(1), 1--3. 10.1175/1520-0493(1950)078<0001:vofeit>2.0.co;2.

See Also

Other Binary Classification Measures: auc(), dor(), fbeta(), fdr(), fnr(), fn(), fomr(), fpr(), fp(), mcc(), npv(), ppv(), prauc(), tnr(), tn(), tpr(), tp()

Examples

# NOT RUN {
set.seed(1)
lvls = c("a", "b")
truth = factor(sample(lvls, 10, replace = TRUE), levels = lvls)
prob = runif(10)
bbrier(truth, prob, positive = "a")
# }