EnvStats (version 2.1.0)

tTestPower: Power of a One- or Two-Sample t-Test

Description

Compute the power of a one- or two-sample t-test, given the sample size, scaled difference, and significance level.

Usage

tTestPower(n.or.n1, n2 = n.or.n1, delta.over.sigma = 0, alpha = 0.05, 
    sample.type = ifelse(!missing(n2), "two.sample", "one.sample"), 
    alternative = "two.sided", approx = FALSE)

Arguments

n.or.n1

numeric vector of sample sizes. When sample.type="one.sample", n.or.n1 denotes $n$, the number of observations in the single sample. When sample.type="two.sample", n.or.n1 denotes $n_1$, the nu

n2

numeric vector of sample sizes for group 2. The default value is the value of n.or.n1. This argument is ignored when sample.type="one.sample". Missing (NA), undefined (NaN), and infinite (

delta.over.sigma

numeric vector specifying the ratio of the true difference ($\delta$) to the population standard deviation ($\sigma$). This is also called the scaled difference. The default value is delta.over.sigma=0.

alpha

numeric vector of numbers between 0 and 1 indicating the Type I error level associated with the hypothesis test. The default value is alpha=0.05.

sample.type

character string indicating whether to compute power based on a one-sample or two-sample hypothesis test. When sample.type="one.sample", the computed power is based on a hypothesis test for a single mean. When sample.type=

alternative

character string indicating the kind of alternative hypothesis. The possible values are "two.sided" (the default), "greater", and "less".

approx

logical scalar indicating whether to compute the power based on an approximation to the non-central t-distribution. The default value is FALSE.

Value

• a numeric vector powers.

Details

If the arguments n.or.n1, n2, delta.over.sigma, and alpha are not all the same length, they are replicated to be the same length as the length of the longest argument. One-Sample Case (sample.type="one.sample") Let $\underline{x} = x_1, x_2, \ldots, x_n$ denote a vector of $n$ observations from a normal distribution with mean $\mu$ and standard deviation $\sigma$, and consider the null hypothesis: $$H_0: \mu = \mu_0 \;\;\;\;\;\; (1)$$ The three possible alternative hypotheses are the upper one-sided alternative (alternative="greater"): $$H_a: \mu > \mu_0 \;\;\;\;\;\; (2)$$ the lower one-sided alternative (alternative="less") $$H_a: \mu < \mu_0 \;\;\;\;\;\; (3)$$ and the two-sided alternative (alternative="two.sided") $$H_a: \mu \ne \mu_0 \;\;\;\;\;\; (4)$$ The test of the null hypothesis (1) versus any of the three alternatives (2)-(4) is based on the Student t-statistic: $$t = \frac{\bar{x} - \mu_0}{s/\sqrt{n}} \;\;\;\;\;\; (5)$$ where $$\bar{x} = \frac{1}{n} \sum_{i=1}^n x_i \;\;\;\;\;\; (6)$$ $$s^2 = \frac{1}{n-1} \sum_{i=1}^n (x_i - \bar{x})^2 \;\;\;\;\;\; (7)$$ Under the null hypothesis (1), the t-statistic in (5) follows a Student's t-distribution with $n-1$ degrees of freedom (Zar, 2010, Chapter 7; Johnson et al., 1995, pp.362-363). The formula for the power of the test depends on which alternative is being tested. The two subsections below describe exact and approximate formulas for the power of the one-sample t-test. Note that none of the equations for the power of the t-test requires knowledge of the values $\delta$ (Equation (12) below) or $\sigma$ (the population standard deviation), only the ratio $\delta/\sigma$. The argument delta.over.sigma is this ratio, and it is referred to as the scaled difference. Exact Power Calculations (approx=FALSE) This subsection describes the exact formulas for the power of the one-sample t-test. Upper one-sided alternative (alternative="greater") The standard Student's t-test rejects the null hypothesis (1) in favor of the upper alternative hypothesis (2) at level-$\alpha$ if $$t \ge t_{\nu}(1 - \alpha) \;\;\;\;\;\; (8)$$ where $$\nu = n - 1 \;\;\;\;\;\; (9)$$ and $t_{\nu}(p)$ denotes the $p$'th quantile of Student's t-distribution with $\nu$ degrees of freedom (Zar, 2010; Berthouex and Brown, 2002). The power of this test, denoted by $1-\beta$, where $\beta$ denotes the probability of a Type II error, is given by: $$1 - \beta = Pr[t_{\nu, \Delta} \ge t_{\nu}(1 - \alpha)] = 1 - G[t_{\nu}(1 - \alpha), \nu, \Delta] \;\;\;\;\;\; (10)$$ where $$\Delta = \sqrt{n} (\frac{\delta}{\sigma}) \;\;\;\;\;\; (11)$$ $$\delta = \mu - \mu_0 \;\;\;\;\;\ (12)$$ and $t_{\nu, \Delta}$ denotes a non-central Student's t-random variable with $\nu$ degrees of freedom and non-centrality parameter $\Delta$, and $G(x, \nu, \Delta)$ denotes the cumulative distribution function of this random variable evaluated at $x$ (Johnson et al., 1995, pp.508-510). Lower one-sided alternative (alternative="less") The standard Student's t-test rejects the null hypothesis (1) in favor of the lower alternative hypothesis (3) at level-$\alpha$ if $$t \le t_{\nu}(\alpha) \;\;\;\;\;\; (13)$$ and the power of this test is given by: $$1 - \beta = Pr[t_{\nu, \Delta} \le t_{\nu}(\alpha)] = G[t_{\nu}(\alpha), \nu, \Delta] \;\;\;\;\;\; (14)$$ Two-sided alternative (alternative="two.sided") The standard Student's t-test rejects the null hypothesis (1) in favor of the two-sided alternative hypothesis (4) at level-$\alpha$ if $$|t| \ge t_{\nu}(1 - \alpha/2) \;\;\;\;\;\; (15)$$ and the power of this test is given by: $$1 - \beta = Pr[t_{\nu, \Delta} \le t_{\nu}(\alpha/2)] + Pr[t_{\nu, \Delta} \ge t_{\nu}(1 - \alpha/2)]$$ $$= G[t_{\nu}(\alpha/2), \nu, \Delta] + 1 - G[t_{\nu}(1 - \alpha/2), \nu, \Delta] \;\;\;\;\;\; (16)$$ The power of the t-test given in Equation (16) can also be expressed in terms of the cumulative distribution function of the non-central F-distribution as follows. Let $F_{\nu_1, \nu_2, \Delta}$ denote a non-central F random variable with $\nu_1$ and $\nu_2$ degrees of freedom and non-centrality parameter $\Delta$, and let $H(x, \nu_1, \nu_2, \Delta)$ denote the cumulative distribution function of this random variable evaluated at $x$. Also, let $F_{\nu_1, \nu_2}(p)$ denote the $p$'th quantile of the central F-distribution with $\nu_1$ and $\nu_2$ degrees of freedom. It can be shown that $$(t_{\nu, \Delta})^2 \cong F_{1, \nu, \Delta^2} \;\;\;\;\;\; (17)$$ where $\cong$ denotes equal in distribution. Thus, it follows that $$[t_{\nu}(1 - \alpha/2)]^2 = F_{1, \nu}(1 - \alpha) \;\;\;\;\;\; (18)$$ so the formula for the power of the t-test given in Equation (16) can also be written as: $$1 - \beta = Pr{(t_{\nu, \Delta})^2 \ge [t_{\nu}(1 - \alpha/2)]^2}$$ $$= Pr[F_{1, \nu, \Delta^2} \ge F_{1, \nu}(1 - \alpha)] = 1 - H[F_{1, \nu}(1-\alpha), 1, \nu, \Delta^2] \;\;\;\;\;\; (19)$$ Approximate Power Calculations (approx=TRUE) Zar (2010, pp.115--118) presents an approximation to the power for the t-test given in Equations (10), (14), and (16) above. His approximation to the power can be derived by using the approximation $$\sqrt{n} (\frac{\delta}{s}) \approx \sqrt{n} (\frac{\delta}{\sigma}) = \Delta \;\;\;\;\;\; (20)$$ where $\approx$ denotes approximately equal to. Zar's approximation can be summarized in terms of the cumulative distribution function of the non-central t-distribution as follows: $$G(x, \nu, \Delta) \approx G(x - \Delta, \nu, 0) = G(x - \Delta, \nu) \;\;\;\;\;\; (21)$$ where $G(x, \nu)$ denotes the cumulative distribution function of the central Student's t-distribution with $\nu$ degrees of freedom evaluated at $x$. The following three subsections explicitly derive the approximation to the power of the t-test for each of the three alternative hypotheses. Upper one-sided alternative (alternative="greater") The power for the upper one-sided alternative (2) given in Equation (10) can be approximated as: $$1 - \beta = Pr[t \ge t_{\nu}(1 - \alpha)]$$ $$= Pr[\frac{\bar{x} - \mu}{s/\sqrt{n}} \ge t_{\nu}(1 - \alpha) - \sqrt{n}\frac{\delta}{s}]$$ $$\approx Pr[t_{\nu} \ge t_{\nu}(1 - \alpha) - \Delta]$$ $$= 1 - Pr[t_{\nu} \le t_{\nu}(1 - \alpha) - \Delta]$$ $$= 1 - G[t_{\nu}(1-\alpha) - \Delta, \nu] \;\;\;\;\;\; (22)$$ where $t_{\nu}$ denotes a central Student's t-random variable with $\nu$ degrees of freedom. Lower one-sided alternative (alternative="less") The power for the lower one-sided alternative (3) given in Equation (14) can be approximated as: $$1 - \beta = Pr[t \le t_{\nu}(\alpha)]$$ $$= Pr[\frac{\bar{x} - \mu}{s/\sqrt{n}} \le t_{\nu}(\alpha) - \sqrt{n}\frac{\delta}{s}]$$ $$\approx Pr[t_{\nu} \le t_{\nu}(\alpha) - \Delta]$$ $$= G[t_{\nu}(\alpha) - \Delta, \nu] \;\;\;\;\;\; (23)$$ Two-sided alternative (alternative="two.sided") The power for the two-sided alternative (4) given in Equation (16) can be approximated as: $$1 - \beta = Pr[t \le t_{\nu}(\alpha/2)] + Pr[t \ge t_{\nu}(1 - \alpha/2)]$$ $$= Pr[\frac{\bar{x} - \mu}{s/\sqrt{n}} \le t_{\nu}(\alpha/2) - \sqrt{n}\frac{\delta}{s}] + Pr[\frac{\bar{x} - \mu}{s/\sqrt{n}} \ge t_{\nu}(1 - \alpha) - \sqrt{n}\frac{\delta}{s}]$$ $$\approx Pr[t_{\nu} \le t_{\nu}(\alpha/2) - \Delta] + Pr[t_{\nu} \ge t_{\nu}(1 - \alpha/2) - \Delta]$$ $$= G[t_{\nu}(\alpha/2) - \Delta, \nu] + 1 - G[t_{\nu}(1-\alpha/2) - \Delta, \nu] \;\;\;\;\;\; (24)$$ Two-Sample Case (sample.type="two.sample") Let $\underline{x}_1 = x_{11}, x_{12}, \ldots, x_{1n_1}$ denote a vector of $n_1$ observations from a normal distribution with mean $\mu_1$ and standard deviation $\sigma$, and let $\underline{x}_2 = x_{21}, x_{22}, \ldots, x_{2n_2}$ denote a vector of $n_2$ observations from a normal distribution with mean $\mu_2$ and standard deviation $\sigma$, and consider the null hypothesis: $$H_0: \mu_1 = \mu_2 \;\;\;\;\;\; (25)$$ The three possible alternative hypotheses are the upper one-sided alternative (alternative="greater"): $$H_a: \mu_1 > \mu_2 \;\;\;\;\;\; (26)$$ the lower one-sided alternative (alternative="less") $$H_a: \mu_1 < \mu_2 \;\;\;\;\;\; (27)$$ and the two-sided alternative (alternative="two.sided") $$H_a: \mu_1 \ne \mu_2 \;\;\;\;\;\; (28)$$ The test of the null hypothesis (25) versus any of the three alternatives (26)-(28) is based on the Student t-statistic: $$t = \frac{\bar{x}_1 - \bar{x}_2}{s_p\sqrt{\frac{1}{n_1} + \frac{1}{n_2}}} \;\;\;\;\;\; (29)$$ where $$\bar{x}_1 = \frac{1}{n_1} \sum_{i=1}^{n_1} x_{1i} \;\;\;\;\;\; (30)$$ $$\bar{x}_2 = \frac{1}{n_2} \sum_{i=1}^{n_2} x_{2i} \;\;\;\;\;\; (31)$$ $$s_p^2 = \frac{(n_1 - 1) s_1^2 + (n_2 - 1) s_2^2}{n_1 + n_2 - 2} \;\;\;\;\;\; (32)$$ $$s_1^2 = \frac{1}{n_1 - 1} \sum_{i=1}^{n_1} (x_{1i} - \bar{x}_1)^2 \;\;\;\;\;\; (33)$$ $$s_2^2 = \frac{1}{n_2 - 1} \sum_{i=1}^{n_2} (x_{2i} - \bar{x}_2)^2 \;\;\;\;\;\; (34)$$ Under the null hypothesis (25), the t-statistic in (29) follows a Student's t-distribution with $n_1 + n_2 - 2$ degrees of freedom (Zar, 2010, Chapter 8; Johnson et al., 1995, pp.508--510, Helsel and Hirsch, 1992, pp.124--128). The formulas for the power of the two-sample t-test are precisely the same as those for the one-sample case, with the following modifications: $$\nu = n_1 + n_2 - 2 \;\;\;\;\;\; (35)$$ $$\Delta = \sqrt{\frac{n_1 n_2}{n_1 + n_2}} (\frac{\delta}{\sigma}) \;\;\;\;\;\; (36)$$ $$\delta = \mu_1 - \mu_2 \;\;\;\;\;\; (37)$$ Note that none of the equations for the power of the t-test requires knowledge of the values $\delta$ or $\sigma$ (the population standard deviation for both populations), only the ratio $\delta/\sigma$. The argument delta.over.sigma is this ratio, and it is referred to as the scaled difference.

References

Berthouex, P.M., and L.C. Brown. (2002). Statistics for Environmental Engineers. Second Edition. Lewis Publishers, Boca Raton, FL. Helsel, D.R., and R.M. Hirsch. (1992). Statistical Methods in Water Resources Research. Elsevier, New York, NY, Chapter 7. Johnson, N. L., S. Kotz, and N. Balakrishnan. (1995). Continuous Univariate Distributions, Volume 2. Second Edition. John Wiley and Sons, New York, Chapters 28, 31 Millard, S.P., and N.K. Neerchal. (2001). Environmental Statistics with S-PLUS. CRC Press, Boca Raton, FL. USEPA. (1989b). Statistical Analysis of Ground-Water Monitoring Data at RCRA Facilities, Interim Final Guidance. EPA/530-SW-89-026. Office of Solid Waste, U.S. Environmental Protection Agency, Washington, D.C. USEPA. (2009). Statistical Analysis of Groundwater Monitoring Data at RCRA Facilities, Unified Guidance. EPA 530/R-09-007, March 2009. Office of Resource Conservation and Recovery Program Implementation and Information Division. U.S. Environmental Protection Agency, Washington, D.C. Zar, J.H. (2010). Biostatistical Analysis. Fifth Edition. Prentice-Hall, Upper Saddle River, NJ.

See Also

tTestN, tTestScaledMdd, tTestAlpha, plotTTestDesign, Normal, t.test, Hypothesis Tests.

Examples

# Look at how the power of the one-sample t-test increases with 
  # increasing sample size:

  seq(5, 30, by = 5) 
  #[1] 5 10 15 20 25 30 

  power <- tTestPower(n.or.n1 = seq(5, 30, by = 5), delta.over.sigma = 0.5) 

  round(power, 2) 
  #[1] 0.14 0.29 0.44 0.56 0.67 0.75

  #----------

  # Repeat the last example, but use the approximation.
  # Note how the approximation underestimates the power 
  # for the smaller sample sizes.
  #----------------------------------------------------

  power <- tTestPower(n.or.n1 = seq(5, 30, by = 5), delta.over.sigma = 0.5, 
    approx = TRUE) 

  round(power, 2) 
  #[1] 0.10 0.26 0.42 0.56 0.67 0.75

  #----------

  # Look at how the power of the two-sample t-test increases with increasing 
  # scaled difference:

  seq(0.5, 2, by = 0.5) 
  #[1] 0.5 1.0 1.5 2.0 

  power <- tTestPower(10, sample.type = "two.sample", 
    delta.over.sigma = seq(0.5, 2, by = 0.5)) 

  round(power, 2) 
  #[1] 0.19 0.56 0.89 0.99

  #----------

  # Look at how the power of the two-sample t-test increases with increasing values 
  # of Type I error:

  power <- tTestPower(20, sample.type = "two.sample", delta.over.sigma = 0.5, 
    alpha = c(0.001, 0.01, 0.05, 0.1)) 

  round(power, 2) 
  #[1] 0.03 0.14 0.34 0.46

  #==========

  # Modifying the example on pages 21-4 to 21-5 of USEPA (2009), determine how 
  # adding another four months of observations to increase the sample size from 
  # 4 to 8 for any one particular compliance well will affect the power of a 
  # one-sample t-test that compares the mean for the well with the MCL of 
  # 7 ppb.  Use alpha = 0.01, assume an upper one-sided alternative 
  # (i.e., compliance well mean larger than 7 ppb), and assume a scaled 
  # difference of 2.  (The data are stored in EPA.09.Ex.21.1.aldicarb.df.) 
  # Note that the power changes from 49% to 98% by increasing the sample size 
  # from 4 to 8.

  tTestPower(n.or.n1 = c(4, 8), delta.over.sigma = 2, alpha = 0.01, 
    sample.type = "one.sample", alternative = "greater")
  #[1] 0.4865800 0.9835401

  #==========

  # Clean up
  #---------
  rm(power)