R package meta is a user-friendly general package providing standard methods for meta-analysis and supporting Schwarzer et al. (2015), https://link.springer.com/book/10.1007/978-3-319-21416-0.
Guido Schwarzer guido.schwarzer@uniklinik-freiburg.de
R package meta (Schwarzer, 2007; Balduzzi et al., 2019) provides the following statistical methods for meta-analysis.
Common effect (also called fixed effect) and random effects model:
Meta-analysis of continuous outcome data (metacont)
Meta-analysis of binary outcome data (metabin)
Meta-analysis of incidence rates (metainc)
Generic inverse variance meta-analysis (metagen)
Meta-analysis of single correlations (metacor)
Meta-analysis of single means (metamean)
Meta-analysis of single proportions (metaprop)
Meta-analysis of single incidence rates (metarate)
Several plots for meta-analysis:
Forest plot (forest.meta, forest.metabind)
Funnel plot (funnel.meta)
Galbraith plot / radial plot (radial.meta)
L'Abbe plot for meta-analysis with binary outcome data
(labbe.metabin, labbe.default)
Baujat plot to explore heterogeneity in meta-analysis
(baujat.meta)
Bubble plot to display the result of a meta-regression
(bubble.metareg)
Three-level meta-analysis model (Van den Noortgate et al., 2013)
Generalised linear mixed models (GLMMs) for binary and count
data (Stijnen et al., 2010) (metabin,
metainc, metaprop, and
metarate)
Various estimators for the between-study variance
\(\tau^2\) in a random effects model (Veroniki et al., 2016);
see description of argument method.tau below
Hartung-Knapp method for random effects meta-analysis
(Hartung & Knapp, 2001a,b), see description of arguments
method.random.ci and adhoc.hakn.ci below
Kenward-Roger method for random effects meta-analysis
(Partlett and Riley, 2017), see description of arguments
method.random.ci and method.predict below
Prediction interval for the treatment effect of a new study
(Higgins et al., 2009; Partlett and Riley, 2017; Nagashima et al.,
2019), see description of argument method.predict below
Statistical tests for funnel plot asymmetry
(metabias.meta, metabias.rm5) and
trim-and-fill method (trimfill.meta,
trimfill.default) to evaluate bias in meta-analysis
Meta-regression (metareg)
Cumulative meta-analysis (metacum) and
leave-one-out meta-analysis (metainf)
Import data from RevMan Web (read.cdir), RevMan
5 (read.rm5), see also metacr to
conduct meta-analysis for a single comparison and outcome from a
Cochrane review
R package meta provides two vignettes:
vignette("meta-workflow") with an overview of main
functions,
vignette("meta-tutorial") with up-to-date commands for
Balduzzi et al. (2019).
Additional statistical meta-analysis methods are provided by add-on R packages:
Frequentist methods for network meta-analysis (R package netmeta)
Statistical methods for sensitivity analysis in meta-analysis (R package metasens)
Statistical methods for meta-analysis of diagnostic accuracy studies with several cutpoints (R package diagmeta)
In the following, more details on available and default statistical
meta-analysis methods are provided and R function
settings.meta is briefly described which can be used
to change the default settings. Additional information on
meta-analysis objects and available summary measures can be found
on the help pages meta-object and
meta-sm.
The following methods are available in all meta-analysis functions to estimate the between-study variance \(\tau^2\).
| Argument | Method |
method.tau = "REML" | Restricted maximum-likelihood estimator (Viechtbauer, 2005) |
| (default) | |
method.tau = "PM" | Paule-Mandel estimator (Paule and Mandel, 1982) |
method.tau = "DL" | DerSimonian-Laird estimator (DerSimonian and Laird, 1986) |
method.tau = "ML" | Maximum-likelihood estimator (Viechtbauer, 2005) |
method.tau = "HS" | Hunter-Schmidt estimator (Hunter and Schmidt, 2015) |
method.tau = "SJ" | Sidik-Jonkman estimator (Sidik and Jonkman, 2005) |
method.tau = "HE" | Hedges estimator (Hedges and Olkin, 1985) |
method.tau = "EB" | Empirical Bayes estimator (Morris, 1983) |
For GLMMs, only the maximum-likelihood method is available.
Historically, the DerSimonian-Laird method was the de facto standard to estimate the between-study variance \(\tau^2\) and is the default in some software packages including Review Manager 5 (RevMan 5) and R package meta, version 4 and below. However, its role has been challenged and especially the REML and Paule-Mandel estimators have been recommended (Veroniki et al., 2016; Langan et al., 2019). Accordingly, the currenct default in R package meta is the REML estimator.
The following R command could be used to employ the Paule-Mandel instead of the REML estimator in all meta-analyses of the current R session:
settings.meta(method.tau = "PM")
Other estimators for \(\tau^2\) could be selected in a similar way.
Note, for binary outcomes, two variants of the DerSimonian-Laird
estimator are available if the Mantel-Haenszel method is used for
pooling. If argument Q.Cochrane = TRUE (default), the
heterogeneity statistic Q is based on the Mantel-Haenszel instead
of the inverse variance estimator under the common effect
model. This is the estimator for \(\tau^2\) implemented in RevMan
5.
The following methods are available in all meta-analysis functions to calculate a confidence interval for the random effects estimate.
| Argument | Method |
method.random.ci = "classic" | Based on standard normal quantile |
| (DerSimonian and Laird, 1986) (default) | |
method.random.ci = "HK" | Method by Hartung and Knapp (2001a/b) |
method.random.ci = "KR" | Kenward-Roger method (Partlett and Riley, 2017) |
DerSimonian and Laird (1986) introduced the classic random effects model using a quantile of the standard normal distribution to calculate a confidence interval for the random effects estimate. This method implicitly assumes that the weights in the random effects meta-analysis are not estimated but given. Particularly, the uncertainty in the estimation of the between-study variance \(\tau^2\) is ignored.
Hartung and Knapp (2001a,b) proposed an alternative method for random effects meta-analysis based on a refined variance estimator for the treatment estimate and a quantile of a t-distribution with k-1 degrees of freedom where k corresponds to the number of studies in the meta-analysis.
The Kenward-Roger method is only available for the REML estimator
(method.tau = "REML") of the between-study variance
\(\tau^2\) (Partlett and Riley, 2017). This method is based on an
adjusted variance estimate for the random effects
estimate. Furthermore, a quantile of a t-distribution with
adequately modified degrees of freedom is used to calculate the
confidence interval.
For GLMMs and three-level models, the Kenward-Roger method is not
available, but a method similar to Knapp and Hartung (2003) is used
if method.random.ci = "HK". For this method, the variance
estimator is not modified, however, a quantile of a
t-distribution with k-1 degrees of freedom is used;
see description of argument test in
rma.glmm and rma.mv.
Simulation studies (Hartung and Knapp, 2001a,b; IntHout et al., 2014; Langan et al., 2019) show improved coverage probabilities of the Hartung-Knapp method compared to the classic random effects method. However, in rare settings with very homogeneous treatment estimates, the Hartung-Knapp variance estimate can be arbitrarily small resulting in a very narrow confidence interval (Knapp and Hartung, 2003; Wiksten et al., 2016). In such cases, an ad hoc variance correction has been proposed by utilising the variance estimate from the classic random effects model with the Hartung-Knapp method (Knapp and Hartung, 2003; IQWiQ, 2022). An alternative ad hoc approach is to use the confidence interval of the classic common or random effects meta-analysis if it is wider than the interval from the Hartung-Knapp method (Wiksten et al., 2016; Jackson et al., 2017).
Argument adhoc.hakn.ci can be used to choose the ad
hoc correction for the Hartung-Knapp (HK) method:
| Argument | Ad hoc method |
adhoc.hakn.ci = "" | no ad hoc correction (default) |
adhoc.hakn.ci = "se" | use variance correction if HK standard error is smaller |
| than standard error from classic random effects | |
| meta-analysis (Knapp and Hartung, 2003) | |
adhoc.hakn.ci = "IQWiG6" | use variance correction if HK confidence interval |
| is narrower than CI from classic random effects model | |
| with DerSimonian-Laird estimator (IQWiG, 2022) | |
adhoc.hakn.ci = "ci" | use wider confidence interval of classic random effects |
| and HK meta-analysis | |
| (Hybrid method 2 in Jackson et al., 2017) |
For GLMMs and three-level models, the ad hoc variance corrections are not available.
The following methods are available in all meta-analysis functions to calculate a prediction interval for the treatment effect in a single new study.
| Argument | Method |
method.predict = "HTS" | Based on t-distribution (Higgins et al., 2009) (default) |
method.predict = "HK" | Hartung-Knapp method (Partlett and Riley, 2017) |
method.predict = "KR" | Kenward-Roger method (Partlett and Riley, 2017) |
method.predict = "NNF" | Bootstrap approach (Nagashima et al., 2019) |
method.predict = "S" | Based on standard normal quantile (Skipka, 2006) |
By default (method.predict = "HTS"), the prediction interval
is based on a t-distribution with k-2 degrees of
freedom where k corresponds to the number of studies in the
meta-analysis, see equation (12) in Higgins et al. (2009).
The Kenward-Roger method is only available for the REML estimator
(method.tau = "REML") of the between-study variance
\(\tau^2\) (Partlett and Riley, 2017). This method is based on an
adjusted variance estimate for the random effects
estimate. Furthermore, a quantile of a t-distribution with
adequately modified degrees of freedom minus 1 is used to calculate
the prediction interval.
The bootstrap approach is only available if R package pimeta
is installed (Nagashima et al., 2019). Internally, the
pima function is called with argument
method = "boot". Argument seed.predict can be used to
get a reproducible bootstrap prediction interval and argument
seed.predict.subgroup for reproducible bootstrap prediction
intervals in subgroups.
The method of Skipka (2006) ignores the uncertainty in the estimation of the between-study variance \(\tau^2\) and thus has too narrow limits for meta-analyses with a small number of studies.
For GLMMs and three-level models, the Kenward-Roger method and the
bootstrap approach are not available, but a method similar to Knapp
and Hartung (2003) is used if method.random.ci = "HK". For
this method, the variance estimator is not modified, however, a
quantile of a t-distribution with k-2 degrees of
freedom is used; see description of argument test in
rma.glmm and rma.mv.
Argument adhoc.hakn.pi can be used to choose the ad
hoc correction for the Hartung-Knapp method:
| Argument | Ad hoc method |
adhoc.hakn.pi = "" | no ad hoc correction (default) |
adhoc.hakn.pi = "se" | use variance correction if HK standard error is smaller |
For GLMMs and three-level models, the ad hoc variance corrections are not available.
The following methods are available in all meta-analysis functions to calculate a confidence interval for \(\tau^2\) and \(\tau\).
| Argument | Method |
method.tau.ci = "J" | Method by Jackson (2013) |
method.tau.ci = "BJ" | Method by Biggerstaff and Jackson (2008) |
method.tau.ci = "QP" | Q-Profile method (Viechtbauer, 2007) |
method.tau.ci = "PL" | Profile-Likelihood method for three-level |
| meta-analysis model (Van den Noortgate et al., 2013) | |
method.tau.ci = "" | No confidence interval |
The first three methods have been recommended by Veroniki et al. (2016). By default, the Jackson method is used for the DerSimonian-Laird estimator of \(\tau^2\) and the Q-profile method for all other estimators of \(\tau^2\).
The Profile-Likelihood method is the only method available for the three-level meta-analysis model.
For GLMMs, no confidence intervals for \(\tau^2\) and \(\tau\) are calculated.
R function settings.meta can be used to change the
previously described and several other default settings for the
current R session.
Some pre-defined general settings are available:
settings.meta("RevMan5")
settings.meta("JAMA")
settings.meta("BMJ")
settings.meta("IQWiG5")
settings.meta("IQWiG6")
settings.meta("geneexpr")
The first command can be used to reproduce meta-analyses from Cochrane reviews conducted with Review Manager 5 (RevMan 5, https://training.cochrane.org/online-learning/core-software/revman) and specifies to use a RevMan 5 layout in forest plots.
The second command can be used to generate forest plots following
instructions for authors of the Journal of the American
Medical Association
(https://jamanetwork.com/journals/jama/pages/instructions-for-authors/). Study
labels according to JAMA guidelines can be generated using
labels.meta.
The third command can be used to generate forest plots in the current layout of the British Medical Journal.
The next two commands implement the recommendations of the Institute for Quality and Efficiency in Health Care (IQWiG), Germany accordinging to General Methods 5 and 6, respectively (https://www.iqwig.de/en/about-us/methods/methods-paper/).
The last setting can be used to print p-values in scientific notation and to suppress the calculation of confidence intervals for the between-study variance.
See settings.meta for more details on these
pre-defined general settings.
In addition, settings.meta can be used to define
individual settings for the current R session. For example, the
following R command specifies the use of Hartung-Knapp and
Paule-Mandel method, and the printing of prediction intervals for
any meta-analysis generated after execution of this command:
settings.meta(method.random.ci = "HK", method.tau =
"PM", prediction = TRUE)
The following data sets are available in R package meta.
| Data set | Description |
Fleiss1993bin | Aspirin after myocardial infarction |
Fleiss1993cont | Mental health treatment on medical utilisation |
Olkin1995 | Thrombolytic therapy after acute myocardial infarction |
Pagliaro1992 | Prevention of first bleeding in cirrhosis |
amlodipine | Amlodipine for work capacity |
caffeine | Caffeine for daytime drowsiness (Cochrane Practice review) |
cisapride | Cisapride in non-ulcer dispepsia |
lungcancer | Smoking example |
smoking | Smoking example |
woodyplants | Elevated CO$_2$ and total biomass of woody plants |
R package metadat has a large collection of meta-analysis data sets.
Balduzzi S, Rücker G, Schwarzer G (2019): How to perform a meta-analysis with R: a practical tutorial. Evidence-Based Mental Health, 22, 153--160
Biggerstaff BJ, Jackson D (2008): The exact distribution of Cochran’s heterogeneity statistic in one-way random effects meta-analysis. Statistics in Medicine, 27, 6093--110
DerSimonian R & Laird N (1986): Meta-analysis in clinical trials. Controlled Clinical Trials, 7, 177--88
Hartung J, Knapp G (2001a): On tests of the overall treatment effect in meta-analysis with normally distributed responses. Statistics in Medicine, 20, 1771--82
Hartung J, Knapp G (2001b): A refined method for the meta-analysis of controlled clinical trials with binary outcome. Statistics in Medicine, 20, 3875--89
Hedges LV & Olkin I (1985): Statistical methods for meta-analysis. San Diego, CA: Academic Press
Higgins JPT, Thompson SG, Spiegelhalter DJ (2009): A re-evaluation of random-effects meta-analysis. Journal of the Royal Statistical Society: Series A, 172, 137--59
Hunter JE & Schmidt FL (2015): Methods of Meta-Analysis: Correcting Error and Bias in Research Findings (Third edition). Thousand Oaks, CA: Sage
IntHout J, Ioannidis JPA, Borm GF (2014): The Hartung-Knapp-Sidik-Jonkman method for random effects meta-analysis is straightforward and considerably outperforms the standard DerSimonian-Laird method. BMC Medical Research Methodology, 14, 25
IQWiG (2022): General Methods: Version 6.1. https://www.iqwig.de/en/about-us/methods/methods-paper/
Jackson D (2013): Confidence intervals for the between-study variance in random effects meta-analysis using generalised Cochran heterogeneity statistics. Research Synthesis Methods, 4, 220--229
Jackson D, Law M, Rücker G, Schwarzer G (2017): The Hartung-Knapp modification for random-effects meta-analysis: A useful refinement but are there any residual concerns? Statistics in Medicine, 36, 3923--34
Knapp G & Hartung J (2003): Improved tests for a random effects meta-regression with a single covariate. Statistics in Medicine, 22, 2693--710
Langan D, Higgins JPT, Jackson D, Bowden J, Veroniki AA, Kontopantelis E, et al. (2019): A comparison of heterogeneity variance estimators in simulated random-effects meta-analyses. Research Synthesis Methods, 10, 83--98
Schwarzer G (2007): meta: An R package for meta-analysis. R News, 7, 40--5
Schwarzer G, Carpenter JR and Rücker G (2015): Meta-Analysis with R (Use-R!). Springer International Publishing, Switzerland
Skipka G (2006): The inclusion of the estimated inter-study variation into forest plots for random effects meta-analysis - a suggestion for a graphical representation [abstract]. XIV Cochrane Colloquium, Dublin, 23-26.
Stijnen T, Hamza TH, Ozdemir P (2010): Random effects meta-analysis of event outcome in the framework of the generalized linear mixed model with applications in sparse data. Statistics in Medicine, 29, 3046--67
Veroniki AA, Jackson D, Viechtbauer W, Bender R, Bowden J, Knapp G, et al. (2016): Methods to estimate the between-study variance and its uncertainty in meta-analysis. Research Synthesis Methods, 7, 55--79
Van den Noortgate W, López-López JA, Marín-Martínez F, Sánchez-Meca J (2013): Three-level meta-analysis of dependent effect sizes. Behavior Research Methods, 45, 576--94
Viechtbauer W (2005): Bias and efficiency of meta-analytic variance estimators in the random-effects model. Journal of Educational and Behavioral Statistics, 30, 261--93
Viechtbauer W (2007): Confidence intervals for the amount of heterogeneity in meta-analysis. Statistics in Medicine, 26, 37--52
Viechtbauer W (2010): Conducting Meta-Analyses in R with the metafor Package. Journal of Statistical Software, 36, 1--48
Wiksten A, Rücker G, Schwarzer G (2016): Hartung-Knapp method is not always conservative compared with fixed-effect meta-analysis. Statistics in Medicine, 35, 2503--15
meta-object, meta-sm