As an alternative to online WAIC, NIMBLE also provides a function,
calculateWAIC
, that can be called on an MCMC object or a matrix of
samples, after running an MCMC. This function does not require that one
set enableWAIC = TRUE
nor WAIC = TRUE
when calling
runMCMC
. The function checks that the necessary variables were
monitored in the MCMC and returns an error if they were not. This function
behaves identically to the calculateWAIC
method of an MCMC object.
Note that to use this function when using nimbleMCMC
one would
need to build the model outside of nimbleMCMC
.
The calculateWAIC
function requires either an MCMC object or a matrix
(or dataframe) of posterior samples plus a model object. In addition, one
can provide optional burnin
and thin
arguments.
In addition, for compatibility with older versions of NIMBLE (prior to
v0.12.0), one can also use the calculateWAIC
method of the MCMC
object to calculate WAIC after all sampling has been completed.
The calculateWAIC()
method accepts a single argument, nburnin
,
equivalent to the nburnin
argument of the calculateWAIC
function described above.
The calculateWAIC
method can only be used if the enableWAIC
argument to configureMCMC
or to buildMCMC
is set to TRUE
,
or if the NIMBLE option enableWAIC
is set to TRUE
. If a user
attempts to call calculateWAIC
without having set
enableWAIC = TRUE
(either in the call to configureMCMC
, or
buildMCMC
, or as a NIMBLE option), an error will occur.
The calculateWAIC
function and method calculate the WAIC based on
Equations 5, 12, and 13 in Gelman et al. (2014) (i.e., using pWAIC2).
Note that there is not a unique value of WAIC for a model. The
calculateWAIC
function and method only provide the conditional WAIC,
namely the version of WAIC where all parameters directly involved in the
likelihood are treated as \(theta\) for the purposes of Equation 5 from
Gelman et al. (2014). As a result, the user must set the MCMC monitors
(via the monitors
argument) to include all stochastic nodes that
are parents of any data nodes; by default the MCMC monitors are only the
top-level nodes of the model. For more detail on the use of different
predictive distributions, see Section 2.5 from Gelman et al. (2014) or
Ariyo et al. (2019).
Also note that WAIC relies on a partition of the observations, i.e.,
'pointwise' prediction. In calculateWAIC
the sum over log pointwise
predictive density values treats each data node as contributing a single
value to the sum. When a data node is multivariate, that data node contributes
a single value to the sum based on the joint density of the elements in the
node. Note that if one wants the WAIC calculation via calculateWAIC
to be based on the joint predictive density for each group of observations
(e.g., grouping the observations from each person or unit in a longitudinal
data context), one would need to use a multivariate distribution for the
observations in each group (potentially by writing a user-defined
distribution).
For more control over and flexibility in how WAIC is calculated, see
help(waic)
.