Various methods are available to assess model fit, to identify outliers and/or influential studies, and for conducting sensitivity analyses (e.g., standardized residuals, Cook's distances, leave-one-out analyses). Advanced techniques for hypothesis tests and obtaining confidence intervals (e.g., for the average effect size or for the model coefficients in a meta-regression model) have also been implemented (e.g., the Knapp and Hartung method, permutation tests).
The package also provides functions for creating forest, funnel, radial (Galbraith), normal quantile-quantile, L'Abbé, and Baujat plots. The presence of funnel plot asymmetry (which may be indicative of publication bias) and its potential impact on the results can be examined via the rank correlation and Egger's regression test and by applying the trim and fill method.
rma.uni function (with alias rma) provides a general framework for fitting such models. The function can be used in conjunction with any of the usual effect size or outcome measures used in meta-analyses (e.g., log odds ratios, log relative risks, risk differences, mean differences, standardized mean differences, raw correlation coefficients, correlation coefficients transformed with Fisher's r-to-z transformation, and so on). For details on these effect size or outcome measures, see the documentation of the escalc function. The notation and models underlying the rma.uni function are explained below. For a set of $i = 1, \ldots, k$ independent studies, let yᵢ denote the observed value of the effect size or outcome measure in the $i$th study. Let \thetaᵢ denote the corresponding (unknown) true effect or outcome, such that yᵢ | \thetaᵢ ~ N(\thetaᵢ, vᵢ). In other words, the observed effects or outcomes are assumed to be unbiased and normally distributed estimates of the corresponding true effects or outcomes with sampling variances equal to vᵢ. The vᵢ values are assumed to be known. Depending on the outcome measure used, a bias correction, normalizing, and/or variance stabilizing transformation may be necessary to ensure that these assumptions are (approximately) true (e.g., the log transformation for odds ratios, the bias correction for standardized mean differences, Fisher's r-to-z transformation for correlations; see escalc for more details). The fixed-effects model conditions on the true effects/outcomes and therefore provides a conditional inference about the set of $k$ studies included in the meta-analysis. When using weighted estimation, this implies that the fitted model provides an estimate of \theta_w = \sum wᵢ \thetaᵢ / \sum wᵢ, that is, the weighted average of the true effects/outcomes in the set of $k$ studies, with weights equal to wᵢ = 1/vᵢ (this is what is often described as the ‘inverse-variance’ method in the meta-analytic literature). One can also employ an unweighted estimation method, which provides an estimate of the unweighted average of the true effects/outcomes in the set of $k$ studies, that is, an estimate of \theta_u = \sum \thetaᵢ / k. For weighted estimation, one could also choose to estimate $\theta_w$, where the wᵢ values are user-defined weights (inverse-variance weights or unity weights as in unweighted estimation are just special cases). It is up to the user to decide to what extent $\theta_w$ is a meaningful parameter to estimate (regardless of the weights used). Moderators can be included in the fixed-effects model, yielding a fixed-effects with moderators model. Again, since the model conditions on the set of $k$ studies included in the meta-analysis, the regression coefficients from the fitted model estimate the weighted relationship (in the least squares sense) between the true effects/outcomes and the moderator variables within the set of $k$ studies included in the meta-analysis (again using weights equal to wᵢ = 1/vᵢ). The unweighted relationship between the true effects/outcomes and the moderator variables can be estimated when using the unweighted estimation method. Again, user-defined weights could also be used. The random-effects model does not condition on the true effects/outcomes. Instead, the $k$ studies included in the meta-analysis are assumed to be a random selection from a larger population of studies. In rare cases, the studies included in a meta-analysis are actually sampled from a larger collection of studies. More typically, the population of studies is a hypothetical population of an essentially infinite set of studies comprising all of the studies that have been conducted, that could have been conducted, or that may be conducted in the future. We assume that \thetaᵢ ~ N(\mu, \tau²), that is, the true effects/outcomes in the population of studies are normally distributed with $\mu$ denoting the average true effect/outcome and \tau² denoting the variance of the true effects/outcomes in the population (\tau² is therefore often referred to as the ‘amount of heterogeneity’ in the true effects/outcomes). The random-effects model can therefore also be written as yᵢ = \mu + uᵢ + eᵢ, where
uᵢ ~ N(0, \tau²) and eᵢ ~ N(0, vᵢ). The fitted model provides an estimate of $\mu$ and \tau². Consequently, the random-effects model provides an unconditional inference about the average true effect/outcome in the population of studies (from which the $k$ studies included in the meta-analysis are assumed to be a random selection). When including moderator variables in the random-effects model, we obtain what is typically called a mixed-effects model in the meta-analytic literature. Such a meta-regression model can also be written as yᵢ = \beta₀ + \beta₁ xᵢ₁ + \beta₂ xᵢ₂ + \ldots + uᵢ + eᵢ, where uᵢ ~ N(0, \tau²) and eᵢ ~ N(0, vᵢ) as before, xᵢ₁ denotes the value of the first moderator variable for the $i$th study, xᵢ₂ denotes the value of the second moderator variable for the $i$th study, and so on (letting $p$ denote the total number of coefficients in the model, including the model intercept if it is included). Therefore, \beta₁ denotes how the average true effect/outcome changes for a one unit increase in xᵢ₁, \beta₂ denotes how the average true effect/outcome changes for a one unit increase in xᵢ₂, and so on, and the model intercept \beta₀ denotes the average true effect/outcome when the values of all moderator variables are equal to zero. The coefficients from the fitted model therefore estimate the relationship between the average true effect/outcome in the population of studies and the moderator variables included in the model. The value of \tau² in the mixed-effects model denotes the ‘amount of residual heterogeneity’ in the true effects/outcomes (i.e., the amount of variability in the true effects//outcomes that is not accounted for by the moderators included in the model). When using weighted estimation in the context of a random-effects model, the model is fitted with weights equal to wᵢ = 1/(\tau² + vᵢ), with \tau² replaced by its estimate (again, this is the standard ‘inverse-variance’ method for random-effects models). One can also choose unweighted estimation in the context of the random-effects model and even any user-defined weights, although the parameter that is estimated (i.e., $\mu$) remains the same regardless of the estimation method and weights used (as opposed to the fixed-effect model case, where the parameter estimated is different for weighted versus unweighted estimation or when using different weights than the standard inverse-variance weights). Since weighted estimation with inverse-variance weights is most efficient, it is usually to be preferred for random-effects models (while in the fixed-effect model case, we must carefully consider whether $\theta_w$ or $\theta_u$ is the more meaningful parameter to estimate). The same principle applies to mixed-effects (versus fixed-effects with moderators) models. Contrary to what is often stated in the literature, it is important to realize that the fixed-effects model does not assume that the true effects/outcomes are homogeneous (i.e., that \thetaᵢ is equal to some common value $\theta$ in all $k$ studies). In other words, fixed-effects models provide perfectly valid inferences under heterogeneity, as long as one is restricting these inferences to the set of studies included in the meta-analysis and one realizes that the model does not provide an estimate of $\theta$, but of $\theta_w$ or $\theta_u$. In the special case that the true effects/outcomes are actually homogeneous (the equal-effects case), the distinction between fixed- and random-effects models disappears, since homogeneity implies that $\mu = \theta_w = \theta_u = \theta$. However, since there is no infallible method to test whether the true effects/outcomes are really homogeneous or not, a researcher should decide on the type of inference desired before examining the data and choose the model accordingly. In fact, there is nothing wrong with fitting both fixed- and random/mixed-effects models to the same data, since these models address different questions. For more details on the distinction between equal-, fixed-, and random-effects models, see Laird and Mosteller (1990) and Hedges and Vevea (1998).rma.mh function. It can be used in combination with odds ratios, relative risks, risk differences, incidence rate ratios, and incidence rate differences. The Mantel-Haenszel method is always based on a weighted estimation approach.rma.peto function. The method provides a weighted estimate of the (log) odds ratio under a fixed-effects model. The method is particularly advantageous when the event of interest is rare, but see the documentation of the function for some caveats.rma.glmm function. These models are essentially special cases of generalized linear (mixed-effects) models (i.e., mixed-effects logistic and Poisson regression models). For $2x2$ table data, a mixed-effects conditional logistic model (based on the non-central hypergeometric distribution) is also available. Random/mixed-effects models with dichotomous data are often referred to as ‘binomial-normal’ models in the meta-analytic literature. Analogously, for event count data, such models could be referred to as ‘Poisson-normal’ models.rma.mv function can be used to fit suitable meta-analytic multivariate/multilevel models to such data, so that the non-independence in the observed/true effects or outcomes is accounted for. Network meta-analyses (also called multiple/mixed treatment comparison meta-analyses) can also be carried out with this function.metafor.news(), you can read the ‘NEWS’ file of the package after installation. Comments, feedback, and suggestions for improvements are very welcome.vignette("metafor"). In addition to reading the paper, carefully read this page and then the help pages for the escalc and the rma.uni functions (or the rma.mh, rma.peto, rma.glmm, rma.mv functions if you intend to use these methods). The help pages for these functions provide links to many additional functions, which can be used after fitting a model. A (pdf) diagram showing the various functions in the metafor package (and how they are related to each other) can be opened with the command vignette("metafor_diagram"). Finally, additional information about the package, several detailed analysis examples, examples of plots and figures provided by the package (with the corresponding code), some additional tips and notes, and a FAQ can be found on the package website at http://www.metafor-project.org/.Gleser, L. J., & Olkin, I. (2009). Stochastically dependent effect sizes. In H. Cooper, L. V. Hedges, & J. C. Valentine (Eds.), The handbook of research synthesis and meta-analysis (2nd ed., pp. 357--376). New York: Russell Sage Foundation.
Hedges, L. V., & Olkin, I. (1985). Statistical methods for meta-analysis. San Diego, CA: Academic Press.
Hedges, L. V., & Vevea, J. L. (1998). Fixed- and random-effects models in meta-analysis. Psychological Methods, 3, 486--504.
Laird, N. M., & Mosteller, F. (1990). Some statistical methods for combining experimental results. International Journal of Technology Assessment in Health Care, 6, 5--30.
Mantel, N., & Haenszel, W. (1959). Statistical aspects of the analysis of data from retrospective studies of disease. Journal of the National Cancer Institute, 22, 719--748.
Viechtbauer, W. (2010). Conducting meta-analyses in R with the metafor package. Journal of Statistical Software, 36(3), 1--48. http://www.jstatsoft.org/v36/i03/.
Yusuf, S., Peto, R., Lewis, J., Collins, R., & Sleight, P. (1985). Beta blockade during and after myocardial infarction: An overview of the randomized trials. Progress in Cardiovascular Disease, 27, 335--371.