More information regarding Confidence Intervals and how they are computed in
effectsize.
Unless stated otherwise, confidence intervals are estimated using the
Noncentrality parameter method; These methods searches for a the best
non-central parameters (ncps) of the noncentral t-, F- or Chi-squared
distribution for the desired tail-probabilities, and then convert these
ncps to the corresponding effect sizes. (See full effectsize-CIs for
more.)
Keep in mind that ncp confidence intervals are inverted significance tests,
and only inform us about which values are not significantly different than
our sample estimate. (They do not inform us about which values are
plausible, likely or compatible with our data.) Thus, when CIs contain the
value 0, this should not be taken to mean that a null effect size is
supported by the data; Instead this merely reflects a non-significant test
statistic - i.e. the p-value is greater than alpha (Morey et al., 2016).
For positive only effect sizes (Eta squared, Cramer's V, etc.; Effect sizes associated with Chi-squared and F distributions), this applies also to cases where the lower bound of the CI is equal to 0. Even more care should be taken when the upper bound is equal to 0 - this occurs when p-value is greater than 1-alpha/2 making, the upper bound cannot be estimated, and the upper bound is arbitrarily set to 0 (Steiger, 2004). For example:
eta_squared(aov(mpg ~ factor(gear) + factor(cyl), mtcars[1:7, ]))
## # Effect Size for ANOVA (Type I) ## ## Parameter | Eta2 (partial) | 90% CI ## -------------------------------------------- ## factor(gear) | 0.58 | [0.00, 0.84] ## factor(cyl) | 0.46 | [0.00, 0.78]
For very large sample sizes, the width of the CI can be smaller than the tolerance of the optimizer, resulting in CIs of width 0. This can also, result in the estimated CIs excluding the point estimate. For example:
chisq_to_cramers_v(13223.73, n = 76227, nrow = 6, ncol = 1)
## Cramer's V | 95% CI ## ------------------------- ## 0.19 | [0.20, 0.20]
t_to_d(80, df_error = 4555555)
## d | 95% CI ## ------------------- ## 0.07 | [0.08, 0.08]
Morey, R. D., Hoekstra, R., Rouder, J. N., Lee, M. D., & Wagenmakers, E. J. (2016). The fallacy of placing confidence in confidence intervals. Psychonomic bulletin & review, 23(1), 103-123.
Steiger, J. H. (2004). Beyond the F test: Effect size confidence intervals and tests of close fit in the analysis of variance and contrast analysis. Psychological Methods, 9, 164-182.