brierScore: Brier Scores

Description

Computes Brier Scores for climatology, raw ensemble, and ensemble forecasting models given observation thresholds.

Usage

brierScore( fit, ensembleData, thresholds, dates = NULL, ...)

Value

A data frame giving the Brier Scores for climatology (empirical distribution of the verifying observations), ensemble (voting), and ensemble foreacsting models for the specified thresholds.

A logistic Brier score is also given for the BMAgamma0 model.

Arguments

fit: A model fit to ensemble forecasting data.
ensembleData: An ensembleData object including ensemble forecasts, verification observations and possibly dates. Missing values (indicated by NA) are allowed. \ This need not be the data used for the model fit, although it must include the same ensemble members.
thresholds: One or more threshold values for the Brier score computations.
dates: The dates for which the Brier score will be computed. These dates must be consistent with fit and ensembleData. The default is to use all of the dates in fit. The dates are ignored if fit originates from fitBMA, which also ignores date information.
...: Included for generic function compatibility.

References

G. W. Brier, Verification of forecasts expressed in terms of probability, Monthly Weather Review, 78:1--3, 1950.

T. Gneiting and A. E. Raftery, Strictly proper scoring rules, prediction and estimation, Journal of the American Statistical Association 102:359--378, 2007.

C. Fraley, A. E. Raftery, T. Gneiting and J. M. Sloughter, ensembleBMA: An R Package for Probabilistic Forecasting using Ensembles and Bayesian Model Averaging, Technical Report No. 516R, Department of Statistics, University of Washington, 2007 (revised 2010).

Examples

Run this code

  data(ensBMAtest)

  ensMemNames <- c("gfs","cmcg","eta","gasp","jma","ngps","tcwb","ukmo")
  
  obs <- paste("PCP24","obs", sep = ".")
  ens <- paste("PCP24", ensMemNames, sep = ".")

  prcpTestData <- ensembleData( forecasts = ensBMAtest[,ens],
                                dates = ensBMAtest[,"vdate"], 
                                observations = ensBMAtest[,obs], 
                                station = ensBMAtest[,"station"],
                                forecastHour = 48,
                                initializationTime = "00")

if (FALSE)  # R check
  prcpTestFit <- ensembleBMAgamma0( prcpTestData, trainingDays = 30)
 

  hist(prcpTestData$obs)

  brierScore(prcpTestFit, prcpTestData,
             thresholds = seq(from = 0, to = .5, by = .1))