Various approaches to the calculation of a goodness-of-fit measure
for GLM's analogous to R-squared in the ordinary linear model have been
proposed. Generally termed 'pseudo R-squared' measures, they include
variance-based, likelihood-based, and distribution-specific approaches.
Here however, a more straightforward definition is used, which can be
applied to any model for which fitted values of the response variable are
generated: R-squared is calculated as the squared (weighted) correlation
between the observed and fitted values of the response (in the original
units). This is simply the squared version of the correlation measure
advocated by Zheng & Agresti (2000), itself an intuitive measure of
goodness-of-fit describing the predictive power of a model. As the measure
does not depend on any specific error distribution or model estimating
procedure, it is also generally comparable across many different types of
model (Kvalseth 1985). In the case of the ordinary linear model, the
measure equals the more traditional R-squared based on sums of squares.
If adj = TRUE (default), the 'adjusted' R-squared value is also
returned, which provides an estimate of the population - as opposed to
sample - R-squared. This is achieved via an analytical formula which
adjusts R-squared for the 'degrees of freedom' of the model (i.e. the ratio
of observations to parameters). Here, this is calculated via the 'Pratt'
rather than standard 'Ezekiel/Wherry' formula, shown in a previous
simulation to be the most effective of a range of existing formulas at
estimating the population R-squared, across a range of model specification
scenarios (Yin & Fan 2001). Adjusted R-squared can be used to safeguard
against overfitting of the model to the original sample.
If pred = TRUE (default), a 'predicted' R-squared is also returned,
which is calculated via the same formula as for R-squared but using
cross-validated rather than standard fitted values. These are obtained by
dividing model response residuals by the complement of the observation
leverages (diagonals of the hat matrix), then subtracting these inflated
'predicted' residuals from the response variable. This is essentially a
short cut to obtaining out-of-sample predictions, normally arising via a
leave-one-out cross-validation procedure (in a GLM they will not be exactly
equal to such predictions). The resulting R-squared is an estimate of the
R-squared that would occur were the model to be fitted to new data, and
will be lower than the original R-squared, and likely also the adjusted
R-squared - highlighting the loss of explanatory power when predicting new
data. This measure is a variant of an
existing
one, calculated by substituting the 'PRESS' statistic, i.e. the sum of
squares of the predicted residuals (Allen 1974), for the residual sum of
squares in the classic R-squared formula.
For mixed models, the function will, by default, calculate all R-squared
metrics using fitted values incorporating both the fixed and random
effects, thus encompassing all variation captured by the model. This is
equivalent to the 'conditional' R-squared of Nakagawa et al. (2017).
To include only some or no random effects, simply set the appropriate
formula using the argument re.form, which is passed directly to
predict.merMod. If re.form = NA, R-squared is calculated for
the fixed effects only - equivalent to the 'marginal' R-squared of Nakagawa
et al. (2017).
R-squared values produced by this function will always be bounded between
zero (no fit) and one (perfect fit), meaning that any negative values
arising from calculations will be rounded up to zero. Negative values
typically mean that the fit is 'worse' than the null expectation of no
relationship between the variables, which can be difficult to interpret in
practice and in any case usually only occurs in rare situations, such as
where the intercept is suppressed. Hence, for simplicity and ease of
interpretation, values less than zero are presented here as a complete lack
of model fit.