The ma_r_bb, ma_r_ic, and ma_r_ad functions implement bare-bones, individual-correction, and artifact-distribution correction methods for d values, respectively.
The ma_d function is the master function for meta-analyses of d values - it facilitates the computation of bare-bones, artifact-distribution, and individual-correction meta-analyses of correlations for any number of group-wise contrasts and any number of dependent variables.
When artifact-distribution meta-analyses are performed, ma_d will automatically extract the artifact information from a database and organize it into the requested type of artifact distribution object (i.e., either Taylor series or interactive artifact distributions).
ma_d is also equipped with the capability to clean databases containing inconsistently recorded artifact data, impute missing artifacts (when individual-correction meta-analyses are requested), and remove dependency among samples by forming composites or averaging effect sizes and artifacts.
The automatic compositing features in ma_d are employed when sample_ids and/or construct names are provided.
ma_d(d, n1, n2 = NULL, n_adj = NULL, sample_id = NULL,
citekey = NULL, treat_as_r = FALSE, ma_method = c("bb", "ic",
"ad"), ad_type = c("tsa", "int"), correction_method = "auto",
group_id = NULL, group1 = NULL, group2 = NULL,
group_order = NULL, construct_y = NULL, facet_y = NULL,
measure_y = NULL, construct_order = NULL,
wt_type = c("n_effective", "sample_size", "inv_var_mean",
"inv_var_sample", "DL", "HE", "HS", "SJ", "ML", "REML", "EB", "PM"),
correct_bias = TRUE, correct_rel = NULL, correct_rGg = FALSE,
correct_ryy = TRUE, correct_rr = NULL, correct_rr_g = TRUE,
correct_rr_y = TRUE, indirect_rr = NULL, indirect_rr_g = TRUE,
indirect_rr_y = TRUE, rGg = NULL, pi = NULL, pa = NULL,
ryy = NULL, ryy_restricted = TRUE, ryy_type = "alpha",
k_items_y = NULL, uy = NULL, uy_observed = TRUE, sign_rz = NULL,
sign_rgz = 1, sign_ryz = 1, moderators = NULL,
cat_moderators = TRUE, moderator_type = c("simple", "hierarchical",
"none"), supplemental_ads = NULL, data = NULL,
control = control_psychmeta(), ...)ma_d_ad(ma_obj, ad_obj_g = NULL, ad_obj_y = NULL,
correction_method = "auto", use_ic_ads = c("tsa", "int"),
correct_rGg = FALSE, correct_ryy = TRUE, correct_rr_g = TRUE,
correct_rr_y = TRUE, indirect_rr_g = TRUE, indirect_rr_y = TRUE,
sign_rgz = 1, sign_ryz = 1, control = control_psychmeta(), ...)
ma_d_bb(d, n1, n2 = rep(NA, length(d)), n_adj = NULL,
sample_id = NULL, citekey = NULL, wt_type = c("n_effective",
"sample_size", "inv_var_mean", "inv_var_sample", "DL", "HE", "HS", "SJ",
"ML", "REML", "EB", "PM"), correct_bias = TRUE, moderators = NULL,
cat_moderators = TRUE, moderator_type = c("simple", "hierarchical",
"none"), data = NULL, control = control_psychmeta(), ...)
ma_d_ic(d, n1, n2 = NULL, n_adj = NULL, sample_id = NULL,
citekey = NULL, treat_as_r = FALSE, wt_type = c("n_effective",
"sample_size", "inv_var_mean", "inv_var_sample", "DL", "HE", "HS", "SJ",
"ML", "REML", "EB", "PM"), correct_bias = TRUE, correct_rGg = FALSE,
correct_ryy = TRUE, correct_rr_g = FALSE, correct_rr_y = TRUE,
indirect_rr_g = TRUE, indirect_rr_y = TRUE, rGg = NULL,
pi = NULL, pa = NULL, ryy = NULL, ryy_restricted = TRUE,
ryy_type = "alpha", k_items_y = NULL, uy = NULL,
uy_observed = TRUE, sign_rgz = 1, sign_ryz = 1,
moderators = NULL, cat_moderators = TRUE,
moderator_type = c("simple", "hierarchical", "none"),
supplemental_ads_y = NULL, data = NULL,
control = control_psychmeta(), ...)
Vector or column name of observed d values.
Vector or column name of sample sizes.
Vector or column name of sample sizes.
Optional: Vector or column name of sample sizes adjusted for sporadic artifact corrections.
Optional vector of identification labels for samples/studies in the meta-analysis.
Optional vector of bibliographic citation keys for samples/studies in the meta-analysis (if multiple citekeys pertain to a given effect size, combine them into a single string entry with comma delimiters (e.g., "citkey1,citekey2").
Logical scalar determining whether d values are to be meta-analyzed as d values (FALSE; default) or whether they should be meta-analyzed as correlations and have the final results converted to the d metric (TRUE).
Method to be used to compute the meta-analysis: "bb" (barebones), "ic" (individual correction), or "ad" (artifact distribution).
For when ma_method is "ad", specifies the type of artifact distribution to use: "int" or "tsa".
Character scalar or a matrix with group_id levels as row names and construct_y levels as column names.
When ma_method is "ad", select one of the following methods for correcting artifacts: "auto", "meas", "uvdrr", "uvirr", "bvdrr", "bvirr",
"rbOrig", "rb1Orig", "rb2Orig", "rbAdj", "rb1Adj", and "rb2Adj".
(note: "rb1Orig", "rb2Orig", "rb1Adj", and "rb2Adj" can only be used when Taylor series artifact distributions are provided and "rbOrig" and "rbAdj" can only
be used when interactive artifact distributions are provided). See "Details" of ma_d_ad for descriptions of the available methods.
Vector of group comparison IDs (e.g., Treatment1-Control, Treatment2-Control).
The group_id argument supersedes the group1 and group2 arguments.
If group_id is not NULL, the values supplied to the group_order argument must correspond to group_id values.
Vector of group identification labels (e.g., Treatment1, Treatment2, Control)
Optional vector indicating the order in which (1) group1 and group2 values or (2) group_ids should be arranged.
If group_order is NULL, the order of group pairings will be determined internally using alpha-numeric ordering.
Vector of construct names for construct designated as "Y".
Vector of facet names for constructs designated as "Y". Facet names "global", "overall", and "total" are reserved to indicate observations that represent effect sizes that have already been composited or that represent construct-level measurements rather than facet-level measurements. To avoid double-compositing, any observation with one of these reserved names will only be eligible for auto-compositing with other such observations and will not be combined with narrow facets.
Vector of names for measures associated with constructs designated as "Y".
Vector indicating the order in which Y variables should be arranged.
Type of weight to use in the meta-analysis: options are "n_effective" (effective sample size), "sample_size", "inv_var_mean" (inverse variance computed using mean effect size), and "inv_var_sample" (inverse variance computed using sample-specific effect sizes). Supported options borrowed from metafor are "DL", "HE", "HS", "SJ", "ML", "REML", "EB", and "PM" (see metafor documentation for details about the metafor methods).
Logical scalar that determines whether to correct correlations for small-sample bias (TRUE) or not (FALSE).
Optional named vector that supersedes correct_rGg and correct_ryy. Names should correspond to construct names in group_id and construct_y to determine which constructs should be corrected for unreliability.
Logical scalar or vector that determines whether to correct the grouping variable variable for measurement error (TRUE) or not (FALSE).
Logical scalar or vector that determines whether to correct the Y variable for measurement error (TRUE) or not (FALSE).
Optional named vector that supersedes correct_rr_g and correct_rr_y. Names should correspond to construct names in group_id and construct_y to determine which constructs should be corrected for range restriction.
Logical scalar or vector or column name determining whether each d value should be corrected for range restriction in the grouping variable (TRUE) or not (FALSE).
Logical scalar or vector or column name determining whether each d should be corrected for range restriction in Y (TRUE) or not (FALSE).
Optional named vector that supersedes indirect_rr_g and indirect_rr_y. Names should correspond to construct names in group_id and construct_y to determine which constructs should be corrected for indirect range restriction.
Logical vector or column name determining whether each d should be corrected for indirect range restriction in the grouping variable (TRUE) or not (FALSE).
Superseded in evaluation by correct_rr_g (i.e., if correct_rr_g == FALSE, the value supplied for indirect_rr_g is disregarded).
Logical vector or column name determining whether each d should be corrected for indirect range restriction in Y (TRUE) or not (FALSE).
Superseded in evaluation by correct_rr_y (i.e., if correct_rr_y == FALSE, the value supplied for indirect_rr_y is disregarded).
Vector or column name of reliability estimates for X.
Scalar or vector containing the restricted-group proportions of group membership. If a vector, it must either (1) have as many elements as there are d values or (2) be named so as to match with levels of the group_id argument.
Scalar or vector containing the unrestricted-group proportions of group membership (default = .5). If a vector, it must either (1) have as many elements as there are d values or (2) be named so as to match with levels of the group_id argument.
Vector or column name of reliability estimates for Y.
Logical vector or column name determining whether each element of ryy is an incumbent reliability (TRUE) or an applicant reliability (FALSE).
String vector identifying the types of reliability estimates supplied (e.g., "alpha", "retest", "interrater_r", "splithalf"). See the documentation for ma_r for a full list of acceptable reliability types.
Numeric vector identifying the number of items in each scale.
Vector or column name of u ratios for Y.
Logical vector or column name determining whether each element of uy is an observed-score u ratio (TRUE) or a true-score u ratio (FALSE).
Optional named vector that supersedes sign_rgz and sign_ryz. Names should correspond to construct names in group_id and construct_y to determine the sign of each construct's relationship with the selection mechanism.
Sign of the relationship between X and the selection mechanism (for use with bvirr corrections only).
Sign of the relationship between Y and the selection mechanism (for use with bvirr corrections only).
Matrix or column names of moderator variables to be used in the meta-analysis (can be a vector in the case of one moderator).
Logical scalar or vector identifying whether variables in the moderators argument are categorical variables (TRUE) or continuous variables (FALSE).
Type of moderator analysis: "none" means that no moderators are to be used, "simple" means that moderators are to be examined one at a time, "hierarchical" means that all possible combinations and subsets of moderators are to be examined, and "all" means that simple and hierarchical moderator analyses are to be performed.
Named list (named according to the constructs included in the meta-analysis) of supplemental artifact distribution information from studies not included in the meta-analysis. This is a list of lists, where the elements of a list associated with a construct are named like the arguments of the create_ad() function.
Data frame containing columns whose names may be provided as arguments to vector arguments and/or moderators.
Output from the control_psychmeta() function or a list of arguments controlled by the control_psychmeta() function. Ellipsis arguments will be screened for internal inclusion in control.
Further arguments to be passed to functions called within the meta-analysis.
For ma_d_ad only: Meta-analysis object of correlations or d values (regardless of input metric, output metric will be d).
For ma_d_ad only: Artifact-distribution object for the grouping variable (output of the link{create_ad} or link{create_ad_group} functions).
If ma_obj is of the class ma_master (i.e., the output of ma_r or ma_d), the object supplied for
ad_obj_g must be a named list of artifact distributions with names.
corresponding to the "X" constructs in the meta-analyses contained within ma_obj.
For ma_d_ad only: AArtifact-distribution object for the Y variable (output of the create_ad function).
If ma_obj is of the class ma_master, the object supplied for ad_obj_y must be a named list of artifact distributions with names
corresponding to the "Y" constructs in the meta-analyses contained within ma_obj.
For ma_d_ad only: Determines whether artifact distributions should be extracted from the individual correction results in ma_obj.
Only evaluated when ad_obj_g or ad_obj_y is NULL and ma_obj does not contain individual correction results.
Use one of the following commands: tsa to use the Taylor series method or int to use the interactive method.
For ma_d_ic only: List supplemental artifact distribution information from studies not included in the meta-analysis. The elements of this list are named like the arguments of the create_ad() function.
A nested tabular object of the class "ma_psychmeta". Components of output tables for bare-bones meta-analyses:
Pair_ID Unique identification number for each construct-contrast pairing.
group_contrast Name of the variable analyzed as the group-contrast variable.
construct_y Name of the variable analyzed as construct Y.
analysis_id Unique identification number for each analysis.
analysis_type Type of moderator analyses: Overall, Simple Moderator, or Hierarchical Moderator.
k Number of effect sizes meta-analyzed.
N Total sample size of all effect sizes in the meta-analysis.
mean_d Mean observed d value.
var_d Weighted variance of observed d values.
var_e Predicted sampling-error variance of observed d values.
var_res Variance of observed d values after removing predicted sampling-error variance.
sd_d Square root of var_r.
se_d Standard error of mean_d.
sd_e Square root of var_e.
sd_res Square root of var_res.
CI_LL_XX Lower limit of the confidence interval around mean_d, where "XX" represents the confidence level as a percentage.
CI_UL_XX Upper limit of the confidence interval around mean_d, where "XX" represents the confidence level as a percentage.
CR_LL_XX Lower limit of the credibility interval around mean_d, where "XX" represents the credibility level as a percentage.
CR_UL_XX Upper limit of the credibility interval around mean_d, where "XX" represents the credibility level as a percentage.
Components of output tables for individual-correction meta-analyses:
pair_id Unique identification number for each construct-contrast pairing.
group_contrast Name of the variable analyzed as the group-contrast variable.
construct_y Name of the variable analyzed as construct Y.
analysis_id Unique identification number for each analysis.
analysis_type Type of moderator analyses: Overall, Simple Moderator, or Hierarchical Moderator.
k Number of effect sizes meta-analyzed.
N Total sample size of all effect sizes in the meta-analysis.
mean_d Mean observed d value.
var_d Weighted variance of observed d values.
var_e Predicted sampling-error variance of observed d values.
var_res Variance of observed d values after removing predicted sampling-error variance.
sd_d Square root of var_r.
se_d Standard error of mean_d.
sd_e Square root of var_e.
sd_res Square root of var_res.
mean_delta Mean artifact-corrected d value.
var_d_c Variance of artifact-corrected d values.
var_e_c Predicted sampling-error variance of artifact-corrected d values.
var_delta Variance of artifact-corrected d values after removing predicted sampling-error variance.
sd_d_c Square root of var_r_c.
se_d_c Standard error of mean_delta.
sd_e_c Square root of var_e_c.
sd_delta Square root of var_delta.
CI_LL_XX Lower limit of the confidence interval around mean_delta, where "XX" represents the confidence level as a percentage.
CI_UL_XX Upper limit of the confidence interval around mean_delta, where "XX" represents the confidence level as a percentage.
CR_LL_XX Lower limit of the credibility interval around mean_delta, where "XX" represents the credibility level as a percentage.
CR_UL_XX Upper limit of the credibility interval around mean_delta, where "XX" represents the credibility level as a percentage.
Components of output tables for artifact-distribution meta-analyses:
pair_id Unique identification number for each construct-contrast pairing.
group_contrast Name of the variable analyzed as the group-contrast variable.
construct_y Name of the variable analyzed as construct Y.
analysis_id Unique identification number for each analysis.
analysis_type Type of moderator analyses: Overall, Simple Moderator, or Hierarchical Moderator.
k Number of effect sizes meta-analyzed.
N Total sample size of all effect sizes in the meta-analysis.
mean_d Mean observed d value.
var_d Weighted variance of observed d values.
var_e Predicted sampling-error variance of observed d values.
var_art Amount of variance in observed d values that is attributable to measurement-error and range-restriction artifacts.
var_pre Total predicted artifactual variance (i.e., the sum of var_e and var_art).
var_res Variance of observed d values after removing predicted sampling-error variance and predicted artifact variance.
sd_d Square root of var_d.
se_d Standard error of mean_d.
sd_e Square root of var_e.
sd_art Square root of var_art.
sd_pre Square root of var_pre.
sd_res Square root of var_res.
mean_delta Mean artifact-corrected d value.
var_d Weighted variance of observed d values corrected to the metric of delta.
var_e Predicted sampling-error variance of observed d values corrected to the metric of delta.
var_art Amount of variance in observed d values that is attributable to measurement-error and range-restriction artifacts corrected to the metric of delta.
var_pre Total predicted artifactual variance (i.e., the sum of var_e and var_art) corrected to the metric of delta.
var_delta Variance of artifact-corrected d values after removing predicted sampling-error variance and predicted artifact variance.
sd_d Square root of var_d corrected to the metric of delta.
se_d Standard error of mean_d corrected to the metric of delta.
sd_e Square root of var_e corrected to the metric of delta.
sd_art Square root of var_art corrected to the metric of delta.
sd_pre Square root of var_pre corrected to the metric of delta.
sd_delta Square root of var_delta.
CI_LL_XX Lower limit of the confidence interval around mean_delta, where "XX" represents the confidence level as a percentage.
CI_UL_XX Upper limit of the confidence interval around mean_delta, where "XX" represents the confidence level as a percentage.
CR_LL_XX Lower limit of the credibility interval around mean_delta, where "XX" represents the credibility level as a percentage.
CR_UL_XX Upper limit of the credibility interval around mean_delta, where "XX" represents the credibility level as a percentage.
The options for correction_method are:
"auto" Automatic selection of the most appropriate correction procedure, based on the available artifacts and the logical arguments provided to the function. (default)
"meas" Correction for measurement error only.
"uvdrr" Correction for univariate direct range restriction (i.e., Case II). The choice of which variable to correct for range restriction is made using the correct_rr_x and correct_rr_y arguments.
"uvirr" Correction for univariate indirect range restriction (i.e., Case IV). The choice of which variable to correct for range restriction is made using the correct_rr_x and correct_rr_y arguments.
"bvdrr" Correction for bivariate direct range restriction. Use with caution: This correction is an approximation only and is known to have a positive bias.
"bvirr" Correction for bivariate indirect range restriction (i.e., Case V).
"rbOrig" Not recommended: Raju and Burke's version of the correction for direct range restriction, applied interactively. We recommend using "uvdrr" instead.
"rbAdj" Not recommended: Raju and Burke's version of the correction for direct range restriction, applied interactively. Adjusted to account for range restriction in the reliability of the Y variable. We recommend using "uvdrr" instead.
"rb1Orig" Not recommended: Raju and Burke's version of the correction for direct range restriction, applied using their TSA1 method. We recommend using "uvdrr" instead.
"rb1Adj" Not recommended: Raju and Burke's version of the correction for direct range restriction, applied using their TSA1 method. Adjusted to account for range restriction in the reliability of the Y variable. We recommend using "uvdrr" instead.
"rb2Orig" Not recommended: Raju and Burke's version of the correction for direct range restriction, applied using their TSA2 method. We recommend using "uvdrr" instead.
"rb2Adj" Not recommended: Raju and Burke's version of the correction for direct range restriction, applied using their TSA2 method. Adjusted to account for range restriction in the reliability of the Y variable. We recommend using "uvdrr" instead.
Schmidt, F. L., & Hunter, J. E. (2015). Methods of meta-analysis: Correcting error and bias in research findings (3rd ed.). Thousand Oaks, California: SAGE Publications, Inc. Chapter 4.
Law, K. S., Schmidt, F. L., & Hunter, J. E. (1994). Nonlinearity of range corrections in meta-analysis: Test of an improved procedure. Journal of Applied Psychology, 79(3), 425.
Dahlke, J. A., & Wiernik, B. M. (2018). One of these artifacts is not like the others: Accounting for indirect range restriction in organizational and psychological research. Manuscript submitted for review.
Raju, N. S., & Burke, M. J. (1983). Two new procedures for studying validity generalization. Journal of Applied Psychology, 68(3), 382. https://doi.org/10.1037/0021-9010.68.3.382
# NOT RUN {
### Demonstration of ma_d ###
## The 'ma_d' function can compute multi-construct bare-bones meta-analyses:
ma_d(d = d, n1 = n1, n2 = n2, construct_y = construct, data = data_d_meas_multi)
## It can also perform multiple individual-correction meta-analyses:
ma_d(ma_method = "ic", d = d, n1 = n1, n2 = n2, ryy = ryyi,
construct_y = construct, data = data_d_meas_multi)
## And 'ma_d' can also curate artifact distributions and compute multiple
## artifact-distribution meta-analyses:
ma_d(ma_method = "ad", d = d, n1 = n1, n2 = n2,
ryy = ryyi, correct_rr_y = FALSE,
construct_y = construct, data = data_d_meas_multi)
### Demonstration of ma_d_bb ###
## Example meta-analyses using simulated data:
ma_d_bb(d = d, n1 = n1, n2 = n2,
data = data_d_meas_multi[data_d_meas_multi$construct == "Y",])
ma_d_bb(d = d, n1 = n1, n2 = n2,
data = data_d_meas_multi[data_d_meas_multi$construct == "Z",])
### Demonstration of ma_d_ic ###
## Example meta-analyses using simulated data:
ma_d_ic(d = d, n1 = n1, n2 = n2, ryy = ryyi, correct_rr_y = FALSE,
data = data_d_meas_multi[data_d_meas_multi$construct == "Y",])
ma_d_ic(d = d, n1 = n1, n2 = n2, ryy = ryyi, correct_rr_y = FALSE,
data = data_d_meas_multi[data_d_meas_multi$construct == "Z",])
# }
Run the code above in your browser using DataLab