reproducer (version 0.5.2)

NP4GMetaAnalysisSimulation: NP4GMetaAnalysisSimulation

Description

This function simulates data from a family of experiments, where the number of experiments in a family is defined by the parameter Exp. It simulates data from one of four distributions and uses the data to construct four of groups of equal size (GroupSize). Two groups are assigned as control groups and their distribution is based on the parameter, mean, and the parameter, spread. However, the mean and spread for the control group in Block 2 can be adjusted using the parameters BlockEffect and BlockStdAdj respectively. The other two groups are treatment groups and their distribution is based on the mean+diff and the spread parameter, but the distributions can be adjusted using the StdAdj, BlockEffect and BlockStdAdj parameters. The data from each experiment is analysed separately to estimate the non-parametric statistics P-hat, Cliff's d and their variances. In addition, the estimates of the standardized mean difference and the small sample size adjusted standardized mean difference are calculated. The effect size statistics are then meta-analysed using the method specified by the MAMethod parameter. We output both the average non-parametric effect statistics across the Exp experimet analysed as if they arose from a single large experiment and also the results of meta-analysing each non-parametric effect size. We use the standard parametric effect sizes and their meta-analysis as baselines.Tests of significance are one-sided if the mean difference is non-zero. If the mean difference is zero, two-sided tests are used. In addition, the user can force the use of two-sided tests using the parameter AlwaysTwoSidedTests. This should only be used for comparison with results reported in other simulation studies. The alpha parameter determines the significance level used in the tests.

Usage

NP4GMetaAnalysisSimulation(
  mean,
  sd,
  diff,
  GroupSize,
  Exp = 5,
  type = "n",
  alpha = 0.05,
  seed = 123,
  StdAdj = 0,
  BlockEffect = 0,
  BlockStdAdj = 0,
  StdExp = 0,
  MAMethod,
  returnES = FALSE,
  AlwaysTwoSidedTests = FALSE
)

Value

If returnES is FALSE, the function returns the summary meta-analysis summary statistics otherwise the function returns the effect sizes for each experiment

Arguments

mean

The default value used for the group means in the simulated data. It can be any real number including zero.

sd

The default value used for the spread of the control group and the spread of the treatment group in the simulated data. The value must be a real value greater than 0.

diff

mean+diff is the value used for the mean of the treatment group. It can be zero.

GroupSize

is the size of each of the 4 groups comprising one experiment. Groupsize should be an integer of 4 or more

Exp

is the number of experiments being simulated. Exp should be an integer of 2 or more. It defaults to 5.

type

specifies the distribution being simulated. The permitted values are 'n' for the normal distribution, 'l' for the lognormal distribution, 'g' for the gamma distribution and 'lap' for the Laplace distribution. The parameter defaults to 'n'.

alpha

The Type 1 error rate level use for statistical tests (default 0.05).

seed

specifies the seed to be used to initiate the simulation, so the simulation is repeatable. It defaults to 123.

StdAdj

The value used to introduce heterogeneity into the treatment groups variance if required.

BlockEffect

is the effect of having two different blocks

BlockStdAdj

is the variance associated with the Block. If BlockStdAdj is zero it means we are treating the block effect as a fixed effect. If BlockStdAdj>0, we treat the block effect as a random effect and increase the variance of Block 2 data.

StdExp

if non-zero it simulates a random effect between experiments in the same family (default 0).

MAMethod

defines the method used for meta-analysis

returnES

This determines the format of the output. If returnES=FALSE it returns the summary meta-analysis statistics otherwise it returns the effect sizes and their variances for each experiment in the family

AlwaysTwoSidedTests

If this parameter is TRUE, the function always does two-sided tests. IF the parameter is FALSE (i.e., default), the function does two-sided statistical tests if the difference between treatment groups is 0, if the difference is not 0, it does one-sided tests

Author

Barbara Kitchenham and Lech Madeyski

Examples

as.data.frame(
  NP4GMetaAnalysisSimulation(
    mean = 0, sd = 1, diff = 0.8, GroupSize = 5, Exp = 5, type = "n",
    alpha = 0.05, seed = 457, StdAdj = 0, BlockEffect = 0.5,
    BlockStdAdj = 0, StdExp = 0, MAMethod = "FE", returnES = TRUE))
#     MeanExp    VarExp       StdMD       df       tval t.sig Cliffd  ...
# 1  1.0761565 1.3874542  0.91362108 14.42773  2.0429188  TRUE   0.52 ...

as.data.frame(
  NP4GMetaAnalysisSimulation(
    mean = 0, sd = 1, diff = 0.8, GroupSize = 5, Exp = 5, type = "n",
    alpha = 0.05, seed = 457, StdAdj = 0, BlockEffect = 0.5,
    BlockStdAdj = 0, StdExp = 0, MAMethod = "FE", returnES = FALSE))
#  NumExp GroupSize AveCliffd AveCliffdvar AveCliffdsig Avephat Avephatvar...
# 1     5         5     0.384   0.01369387         TRUE   0.692   0.003336...