metrica (version 2.1.0)

MASE: Mean Absolute Scaled Error (MASE)

Description

It estimates the mean absolute error using the naive-error approach for a continuous predicted-observed dataset.

Usage

MASE(
  data = NULL,
  obs,
  pred,
  time = NULL,
  naive_step = 1,
  oob_mae = NULL,
  tidy = FALSE,
  na.rm = TRUE
)

Value

an object of class numeric within a list (if tidy = FALSE) or within a data frame (if tidy = TRUE).

Arguments

data

(Required) argument to call an existing data frame containing the data.

obs

Vector with observed values (numeric).

pred

Vector with predicted values (numeric).

time

(Optional) String with the "name" of the vector containing the time variable to sort observations. "Optional" to ensure an appropriate MASE estimation. Default: NULL, assumes observations are already sorted by time.

naive_step

A positive number specifying how many observed values to recall back in time when computing the naive expectation. Default = 1

oob_mae

A numeric value indicating the out-of-bag (out-of-sample) MAE. By default, an in-sample MAE is calculated using the naive forecast method. See Hyndman & Koehler (2006). Default : NULL.

tidy

Logical operator (TRUE/FALSE) to decide the type of return. TRUE returns a data.frame, FALSE returns a list; Default : FALSE.

na.rm

Logic argument to remove rows with missing values (NA). Default is na.rm = TRUE.

Details

The MASE is especially well suited for time series predictions. It can be used to compare forecast methods on a single series and also to compare forecast accuracy between series.

This metric never gives an infinite or undefined values (unless all observations over time are exactly the same!).

By default, the MASE scales the error based on in-sample MAE from the naive forecast method (random walk). The in-sample MAE is used in the denominator because it is always available and it effectively scales the errors.Since it is based on absolute error, it is less sensitive to outliers compared to the classic MSE.

\(MASE = \frac{1}{n}(\frac{|O_i-P_i|}{ \frac{1}{T-1} \sum^T_{t=2}~|O_t - O_{t-1}| })\)

If available, users may use and out-of-bag error from an independent dataset, which can be specified with the oob_mae arg. and will replace the denominator into the MASE equation.

MASE measures total error (i.e. both lack of accuracy and precision.). The lower the MASE below 1, the better the prediction quality. MASE = indicates no difference between the model and the naive forecast (or oob predictions). MASE > 1 indicate poor performance.

For the formula and more details, see online-documentation

References

Hyndman, R.J., Koehler, A.B. (2006). Another look at measures of forecast accuracy. _ Int. J. Forecast_ tools:::Rd_expr_doi("10.3354/cr030079")

Examples

# \donttest{
set.seed(1)
# Create a fake dataset
X <- rnorm(n = 100, mean = 8, sd = 10)
Y <- rnorm(n = 100, mean = 8.2, sd = 15)
Z <- seq(1, 100, by = 1)

time_data <- as.data.frame(list("observed" = X, "predicted" = Y, "time" = Z))
 
MASE(data = time_data, obs = observed, pred = predicted, time = time)
# }