get_metrics.forecast_sample

For sample-based forecasts, the default scoring rules are:<ul>
<li>"crps" = <code>crps_sample()</code></li>
<li>"overprediction" = <code>overprediction_sample()</code></li>
<li>"underprediction" = <code>underprediction_sample()</code></li>
<li>"dispersion" = <code>dispersion_sample()</code></li>
<li>"log_score" = <code>logs_sample()</code></li>
<li>"dss" = <code>dss_sample()</code></li>
<li>"mad" = <code>mad_sample()</code></li>
<li>"bias" = <code>bias_sample()</code></li>
<li>"ae_median" = <code>ae_median_sample()</code></li>
<li>"se_mean" = <code>se_mean_sample()</code></li>
</ul>

handle-metrics

Facilitate the evaluation of forecasts in a convenient
framework based on data.table. It allows user to to check their forecasts
and diagnose issues, to visualise forecasts and missing data, to transform
data before scoring, to handle missing forecasts, to aggregate scores, and
to visualise the results of the evaluation. The package mostly focuses on
the evaluation of probabilistic forecasts and allows evaluating several
different forecast types and input formats. Find more information about the
package in the Vignettes as well as in the accompanying paper,
<doi:10.48550/arXiv.2205.07090>.

Nikos Bosse

scoringutils

Utilities for Scoring and Assessing Predictions

Sam Abbott

Hugo Gruson

Johannes Bracher

Toshiaki Asakura 

James Mba Azam

Sebastian Funk

Michael Chirico

get_metrics.forecast_sample function

<dl><dt>x</dt>
<dd>A forecast object (a validated data.table with predicted and
observed values, see <code>as_forecast_binary()</code>).</dd>
<dt>select</dt>
<dd>A character vector of scoring rules to select from the list. If
<code>select</code> is <code>NULL</code> (the default), all possible scoring rules are returned.</dd>
<dt>exclude</dt>
<dd>A character vector of scoring rules to exclude from the list.
If <code>select</code> is not <code>NULL</code>, this argument is ignored.</dd>
<dt>...</dt>
<dd>unused</dd></dl>

Arguments

<div style="text-align: left">
 <img src="figures/metrics-sample.png?package=scoringutils&version=2.1.0" style="width:750px;max-width:100%;">
 </div>
 Overview of required input format for sample-based forecasts

Input format

Get default metrics for sample-based forecasts — get_metrics.forecast_sample

<dl>

<dt>x</dt>
<dd>A forecast object (a validated data.table with predicted and
observed values, see <code>as_forecast_binary()</code>).</dd>


<dt>select</dt>
<dd>A character vector of scoring rules to select from the list. If
<code>select</code> is <code>NULL</code> (the default), all possible scoring rules are returned.</dd>


<dt>exclude</dt>
<dd>A character vector of scoring rules to exclude from the list.
If <code>select</code> is not <code>NULL</code>, this argument is ignored.</dd>


<dt>...</dt>
<dd>unused</dd>

</dl>

<div style="text-align: left">
 <img src='figures/metrics-sample.png' style="width:750px;max-width:100%;" />
 </div>
 Overview of required input format for sample-based forecasts

get_metrics.forecast_sample: Get default metrics for sample-based forecasts

Description

Usage

Arguments

Input format

See Also

Examples