Perform approximate leave-one-out cross-validation based on the posterior likelihood using the loo package.
# S3 method for brmsfit
LOO(x, ..., compare = TRUE, reloo = FALSE,
newdata = NULL, re_formula = NULL, allow_new_levels = FALSE,
sample_new_levels = "uncertainty", new_objects = list(), subset = NULL,
nsamples = NULL, pointwise = NULL, nug = NULL, k_threshold = 0.7,
update_args = list(), cores = 1, wcp = 0.2, wtrunc = 3/4)LOO(x, ...)
A fitted model object typically of class brmsfit.
Optionally more fitted model objects.
A flag indicating if the information criteria
of the models should be compared to each other
via compare_ic.
Logical; Indicate whether
reloo should be applied
on problematic observations. Defaults to FALSE.
An optional data.frame for which to evaluate predictions.
If NULL (default), the orginal data of the model is used.
formula containing group-level effects
to be considered in the prediction.
If NULL (default), include all group-level effects;
if NA, include no group-level effects.
A flag indicating if new
levels of group-level effects are allowed
(defaults to FALSE).
Only relevant if newdata is provided.
Indicates how to sample new levels
for grouping factors specified in re_formula.
This argument is only relevant if newdata is provided and
allow_new_levels is set to TRUE.
If "uncertainty" (default), include group-level uncertainty
in the predictions based on the variation of the existing levels.
If "gaussian", sample new levels from the (multivariate)
normal distribution implied by the group-level standard deviations
and correlations. This options may be useful for conducting
Bayesian power analysis.
If "old_levels", directly sample new levels from the
existing levels.
A numeric vector specifying
the posterior samples to be used.
If NULL (the default), all samples are used.
Positive integer indicating how many
posterior samples should be used.
If NULL (the default) all samples are used.
Ignored if subset is not NULL.
A flag indicating whether to compute the full
log-likelihood matrix at once or separately for each observation.
The latter approach is usually considerably slower but
requires much less working memory. Accordingly, if one runs
into memory issues, pointwise = TRUE is the way to go.
By default, pointwise is automatically chosen based on
the size of the model.
Small positive number for Gaussian process terms only.
For numerical reasons, the covariance matrix of a Gaussian
process might not be positive definite. Adding a very small
number to the matrix's diagonal often solves this problem.
If NULL (the default), nug is chosen internally.
The threshold at which pareto \(k\)
estimates are treated as problematic. Defaults to 0.7.
Only used if argument reloo is TRUE.
See pareto_k_ids
for more details.
A list of further arguments passed to
update.brmsfit such
as iter, chains, or cores.
The number of cores to use for parallelization.
Default is 1.
Parameters used for
the Pareto smoothed importance sampling.
See loo for details.
If just one object is provided, an object of class ic.
If multiple objects are provided, an object of class iclist.
brmsfit: LOO method for brmsfit objects
When comparing models fitted to the same data,
the smaller the LOO, the better the fit.
For brmsfit objects, loo is an alias of LOO.
Use method add_ic to store
information criteria in the fitted model object for later usage.
Vehtari, A., Gelman, A., & Gabry J. (2016). Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC. In Statistics and Computing, doi:10.1007/s11222-016-9696-4. arXiv preprint arXiv:1507.04544.
Gelman, A., Hwang, J., & Vehtari, A. (2014). Understanding predictive information criteria for Bayesian models. Statistics and Computing, 24, 997-1016.
Watanabe, S. (2010). Asymptotic equivalence of Bayes cross validation and widely applicable information criterion in singular learning theory. The Journal of Machine Learning Research, 11, 3571-3594.
# NOT RUN {
# model with population-level effects only
fit1 <- brm(rating ~ treat + period + carry,
data = inhaler, family = "gaussian")
LOO(fit1)
# model with an additional varying intercept for subjects
fit2 <- brm(rating ~ treat + period + carry + (1|subject),
data = inhaler, family = "gaussian")
# compare both models
LOO(fit1, fit2)
# }
# NOT RUN {
# }
Run the code above in your browser using DataLab