spaMM (version 4.5.0)

predVar: Prediction and response variances

Description

spaMM allows computation of four variance components of prediction, returned by predict as “...Var” attributes: predVar, fixefVar, residVar, or respVar. The phrase “prediction variance” is used inconsistently in the literature. Often it is used to denote the uncertainty in the response (therefore, including the residual variance), but spaMM follows some literature for mixed models in departing from this usage. Here, this uncertainly is called the response variance (respVar), while prediction variance (predVar) is used to denote the uncertainty in the linear predictor (as in Booth & Hobert, 1998; see also Jeske & Harville, 1988). The respVar is the predVar plus the residual variance residVar.

Which components are returned is controlled in particular by the type and variances arguments of the relevant functions. variances is a list of booleans whose possible elements either match the possible returned components: predVar, fixefVar, residVar, or respVar; or may additionally include linPred, disp, cov, as_tcrossfac_list and possibly other cryptic ones.

The predict default value for all elements is NULL, which jointly translate to no component being computed, equivalently to setting all elements to FALSE. However, setting one component to TRUE may reverse the default effect for other components. In particular, by default, component predVar implies linPred=TRUE, disp=TRUE and component respVar additionally implies residVar=TRUE; in both cases, the linPred=TRUE default by default implies fixefVar=TRUE. Calling for one variance may imply that some of its components are not only computed but also returned as a distinct attribute.

By default the returned components are vectors of variances (with exceptions for some type value). To obtain covariance matrices (when applicable), set cov=TRUE. as_tcrossfac_list=TRUE can be used to return a list of matrices \(X_i\) such that the predVar covariance matrix equals \(\sum_i X_i X'_i\). It thus provides a representation of the predVar that may be useful in particular when the predVar has large dimension, as the component \(X_i\)s may require less memory (being possibly non-square or sparse).

residVar=TRUE evaluates residVar the residual variance. For families without a dispersion parameter (e.g., binomial or poisson), this is as given by the variance function of the family object (in the binomial case, it is thus the variance of a single binary draw). For families with a dispersion parameter (such as \(\phi\) for gaussian or Gamma families, negative-binomial, beta), it is the residual variance as function of the dispersion parameter, whether this parameter is a single scalar or follows a more complex residual-dispersion model. Prior weights are however ignored (see the residVar etractor for the opposite feature). For the beta-binomial family, it is also the variance of a single binary draw; although this family has a residual-dispersion parameter the latter variance is not affected by it.

fixefVar=TRUE evaluates fixefVar, the variance due to uncertainty in fixed effects (X\(\beta\)).

Computations implying linPred=TRUE will take into account the variances of the linear predictor \(\eta\), i.e. the uncertainty in fixed effects (X\(\beta\)) and random effects (ZLv), for given dispersion parameters (see Details). For fixed-effect models, the fixefVar calculations reduces to the linPred one.

Computations implying disp=TRUE additionally include the effect of uncertainty in estimates of dispersion parameters (\(\lambda\) and \(\phi\)), with some limitations (see Details). variances=list(predVar=TRUE), which evaluates the uncertainty of linear predictor, implies disp=TRUE by default, meaning that it includes such effects of uncertainty in dispersion parameters on the linear predictor. variances=list(respVar=TRUE) performs similarly but additionally includes the residual variance in the returned variance.

Details

fixefVar is the (co)variance of X\(\beta\), deduced from the asymptotic covariance matrix of \(\beta\) estimates.

linPred is the prediction (co)variance of \(\eta\)=X\(\beta\)+Zv (see HLfit Details for notation, and keep in mind that new matrices may replace the ones from the fit object when newdata are used), by default computed for given dispersion parameters. It takes into account the joint uncertainty in estimation of \(\beta\) and prediction of v. In particular, for new levels of the random effects, predVar computation takes into account uncertainty in prediction of v for these new levels. For prediction covariance with a new Z, it matters whether a single or multiple new levels are used: see Examples.

For computations implying disp=TRUE, prediction variance may also include a term accounting for uncertainty in \(\phi\) and \(\lambda\), computed following Booth and Hobert (1998, eq. 19). This computation acheives its originally described purpose for a scalar residual variance (\(\phi\)) and for several random effects with scalar variances (\(\lambda\)). This computation ignores uncertainties in spatial correlation parameters.

The (1998) formulas for the effect of uncertainty in dispersion parameters are here also applied to the variance parameters in random-coefficient terms, but with a one-time warning. Not only this is not expected to account for the uncertainty of the correlation parameter(s) of such terms, but the result will depend on the internal representation (the “square root”) of the covariance matrix, which may differ among the different fitting algorithms that may be used by spaMM.

respVar is the sum of predVar (pre- and post-multiplied by \(\partial\mu/\partial\eta\) for models with non-identity link) and of residVar.

These variance calculations are approximate except for LMMs, and cannot be guaranteed to give accurate results.

References

Booth, J.G., Hobert, J.P. (1998) Standard errors of prediction in generalized linear mixed models. J. Am. Stat. Assoc. 93: 262-272.

Jeske, Daniel R. & Harville, David A. (1988) Prediction-interval procedures and (fixed-effects) confidence-interval procedures for mixed linear models. Communications in Statistics - Theory and Methods, 17: 1053-1087. tools:::Rd_expr_doi("10.1080/03610928808829672")

Examples

if (FALSE) {
# (but run in help("get_predVar"))
data("blackcap")
fitobject <- fitme(migStatus ~ 1 + Matern(1|longitude+latitude),data=blackcap,
                       fixed=list(nu=4,rho=0.4,phi=0.05))

#### multiple controls of prediction variances
# (1) fit with an additional random effect
grouped <- cbind(blackcap,grp=c(rep(1,7),rep(2,7))) 
fitobject <- fitme(migStatus ~ 1 +  (1|grp) +Matern(1|longitude+latitude),
                       data=grouped,  fixed=list(nu=4,rho=0.4,phi=0.05))

# (2) re.form usage to remove a random effect from point prediction and variances: 
predict(fitobject,re.form= ~ 1 +  Matern(1|longitude+latitude))

# (3) comparison of covariance matrices for two types of new data
moregroups <- grouped[1:5,]
rownames(moregroups) <- paste0("newloc",1:5)
moregroups$grp <- rep(3,5) ## all new data belong to an unobserved third group 
cov1 <- get_predVar(fitobject,newdata=moregroups,
                     variances=list(linPred=TRUE,cov=TRUE))
moregroups$grp <- 3:7 ## all new data belong to distinct unobserved groups
cov2 <- get_predVar(fitobject,newdata=moregroups,
                     variances=list(linPred=TRUE,cov=TRUE))
cov1-cov2 ## the expected off-diagonal covariance due to the common group in the first fit.

}
## see help("get_predVar") for further examples