orddom: Ordinal Dominance Statistics

Description

Returns an array of ordinal dominance statistics based on the input of two 1-column matrices as an alternative to independent or paired group mean comparisons (especially for Cliff's delta statistics).

Usage

orddom(x,y,alpha=.05,paired=FALSE,outputfile="",studdist=TRUE,
          symmetric=FALSE,onetailed=FALSE,t.welch=TRUE,
		  x.name="",y.name="",description="")

Arguments

A 1-column matrix with optional column name containing all $n_x$ values or scores of group X or 1 (e.g. control or pretest group.), e.g. declared in R as x<-t(matrix(c($x_1,x_2,x_3,x_4, \ldots , x_{n_x}$),1)) colnames(x)<-c("[label of group X]").

A 1-column matrix with optional column name containing all $n_y$ values of group Y or 2 (e.g. experimental or post-test group). For paired comparisons (e.g. pre-post), $n_x = n_y$ is required. If y is a vector, a default column name is assigned.

alpha

Significance or $\alpha$-level used for the calculation of the confidence intervals. Default value is $\alpha = .05$ or 5 Percent, giving a 95 Percent CI. For multiple dominance comparisons, a Bonferroni procedure may be implemented: Cliff (1996, p.150) s

paired

By default, independence of the two groups or data sets is assumed. If the number of cases in x and y are equal and paired (e.g. pre-post) comparisons, this should be set to TRUE to return the full array of within, between, combined and metric delta stati

outputfile

If a a detailed report of the ordinal dominance analysis is wanted, a filename should be given here. The report as standard text file is written to the current working directory.

studdist

By default, it is assumed that small samples are being examined. In this case, z-values based on Student's t-distribution are used for estimating upper and lower limits of the confidence intervals (CI) as well as z-probabilities. If larger sample sizes a

symmetric

By default, asymmetric confidence intervals (CI) are being calculated to compensate for positive correlations between the samples as generally recommended by the literature on the delta statistics. To increase power in certain cases, however - e.g. in sma

onetailed

By default, calculation of p values and confidence intervals (CI) assumes two-sided testing against the null hypothesis. Set to TRUE if the alternative hypothesis targets at one-tailed testing.

t.welch

By default, for calculation of the t-test scores and metric p and df values, the Welch approximation is used. If set to FALSE, equal variances are assumed for groups X and Y and a pooled variance is being calculated.

x.name

By default, the label of group x (i.e. 1st or control or pretest group) is taken from the column name of the x input matrix. This argument allows for assigning an alternative label.

y.name

This argument allows for assigning an alternative label for the y input matrix or group y (i.e. 2nd or experimental or posttest group).

description

This argument allows for assigning a string (as title or description) for the ordinal comparison outputs.

Value

INDEPENDENT GROUPS (paired argument set to FALSE) In the case of independent groups or data sets X and Y (e.g. comparison group X vs. treatment group Y), a 2-column-matrix containing 29 rows with values is returned. The ordinal statistics can be retrieved from the first column (named "ordinal") while the second column (named "metric") contains metric comparison data where appropriate.
[1 or ["var1_X", col#]Label assigned to group x (x.name or column name of the x input matrix) or a default "1st var (x)".
[2 or ["var2_Y", col#]Label assigned to group x (x.name or column name of the x input matrix) or a default "2nd var (y)".
[3 or ["type_title", col#]Column 1: Returns type of the comparison, in this case "indep". Column2: In case a string header is defined by use of the comp.name argument, it is returned in column 2.
[4 or ["n in X", col#]Number of cases in x (i.e. group X sample size).
[5 or ["n in Y", col#]Number of cases in y (i.e. group Y sample size).
[6 or ["N #Y>X", col#]Number of occurences of an observation from group y having a higher value than an observation from group x when comparing all x scores with all y scores: $N_{\#Y>X}=\#(y_i>x_j) ,$ where # denotes "the number of times" whilst comparing each $i=1, 2, 3, \ldots n_y$ score in sample Y with each $j=1, 2, 3, \ldots n_x$ score in sample X (resulting in $n_x \cdotp n_y$ comparisons).
[7 or ["N #Y=X", col#]Number of occurences of an observation from group y having the same value as an observation from group x: $N_{\#Y=X}=\#(y_i=x_j)$.
[8 or ["N #Ycol#]Number of occurences of an observation from group y having a smaller value than an observation from group x: $N_{\#Y
[9 or ["PS X>Y", col#]Common Language CL effect size or Probability of Superiority (PS) of X over Y, see below.
[10 or ["PS Y>X", col#]Column 1: Discrete case Common Language CL effect size or Probability of Superiority (PS) of Y over X,$PS(Y>X)=\frac{\#(y_i>x_j)}{n_y \cdotp n_x}$ (cf. Grissom, 1994,Grissom & Kim, 2005,McGraw & Wong, 1992). This effect size reflects the probability that a subject or case randomly chosen from group Y has a higher score than than a randomly chosen subject or case from group X (cf. Acion et al., 2006). Column 2: Assuming equal variances and population normality, the (para)metric version of the Common Language effect size is calculated as suggested by McGraw & Wong (1992, p. 361) as $PS(Y>X)=\Phi (\frac{M_y-M_x}{\sqrt{s_x^{2} + s_y^{2}}})$ where $\Phi$ is the cumulative normal distribution function with $\Phi(z_{\alpha})=\alpha$.
[11 or ["A X>Y", col#]Vargha and Delaney's A as stochastic superiority of X over Y, calculated as $$A_{X>Y} = PS(X>Y)+.5 PS(X=Y)$$ (cf. Vargha & Delaney, 1998, 2000,Delaney & Vargha, 2002). This modified probability of superiority effect size has also been called area under the the receiver operating characteristic curve or AUC by Kraemer and Kupfer (2006). If one sampled one single case or subject from group Y and one from group X, respectively, A or AUC is the probability that the sample taken from group Y has a higher score or value than the one sampled from X (given the toss of a coin to break any ties). See also code{dmes} of this package.
[12 or ["A Y>X", col#]Vargha and Delaney's A as stochastic superiority of Y over X.
[13 or ["delta", col#]For column 1 ("ordinal"): Cliff's delta for independent groups (Cliff, 1996,Long et al., 2003):$$d=\frac{\#(y_i>x_j)-\#(y_isuccess rate difference (SRD) effect size by Kraemer and Kupfer, delta denotes the difference between the probability that a randomly chosen Y case or subject (or patient) has a higher score than a randomly chosen case or subject from group X and the probability for the opposite. Put in simple terms, if higher values reflect better treatment outcomes of study participants, delta is the difference between the probability that a Y treatment group participant has a treatment outcome preferable to an X control group participant and the probability that a X patient has a treatment outcome preferable to a Y patient (cf. Kraemer & Kupfer, 2006, p. 994). In contrast to the PS and A effect sizes, delta thus takes potentially worse or harmful treatment outcomes into account. In column 2, the metric differences between the means are given: $\bar{y}-\bar{x}=\bar{d_{ij}}$ between all comparable x and y scores with $d_{ij}=y_i-x_j$.
[14 or ["1-alpha", col#]Significance or $\alpha$-level for CI estimation, given as percentage between 0 and 100.
[15 or ["CI low", col#]Unless the default symmetric parameter is explicitly set to TRUE, improved formulas are used (Feng & Cliff, 2004) to caculate asymmetric confidence interval (CI) boundary estimates of delta or mean difference: $$CI_{lower/upper}=\frac{d - d^{3} \pm t_{\alpha/2} s_{d} \sqrt{1 - 2 d^{2} + d^{4} + t_{\alpha/2}^{2} s_{d}^{2} )}}{1 - d^{2} + t_{\alpha/2}^{2} s_{d}^{2}},$$ with t-values at the given $\alpha$-level taken from Student's t distribution by default (unless the studdist is set FALSE, in which case t-values are based on z-values from the Standard Normal Distribution ). In case the symmetric argument is explicitly set to TRUE, however, ordinary CIs are being calculated with $CI_{lower/upper}=d \pm t_{\alpha/2} s_{d}$. In any case, if Cliffs' $d = \pm 1$, one CI is assumed being equal to d, the respective other is calculated as $$CI_{lower/upper}=((n_{b}-t_{\alpha/2}^{2}))(n_{b}+t_{\alpha/2}^{2})^{-1},$$ where $t_{\alpha/2}$ is the t-value or z-score at the selected $\alpha$ level (2-tailed) of the respective studdist-controlled distribution, and $n_{b}$ the number of observations or cases in the smaller of the two samples.
[16 or ["CI high", col#]Confidence interval upper boundary estimate of delta or mean difference.
[17 or ["s delta", col#]Unbiased sample estimate of the delta standard deviation in column 1. In column 2 ("metric"): Pooled standard deviation of metric mean difference with $s_{xy}=[((n_x-1)s_x + (n_y-1)s_y)/(n_x+n_y-2)]^{1/2}$ .
[18 or ["var delta", col#]Column 1: Variance of delta (unbiased sample estimate), calculated as $$s_{d}^{2} = \frac{n_y^{2} \sum(d_{i\cdot} - d)^{2} + n_{x}^{2} \sum(d_{\cdot j} - d)^{2} - \sum \sum (d_{ij} - d)^{2}}{n_{x} n_{y} (n_{x}-1) (n_{y}-1)},$$ or, using the partial variances $$s_{d}^{2}= \frac{n_y^{2}(n_x-1)s_{d_{i \cdot}}^2 + n_x^{2} (n_y-1) s_{d_{\cdot j}}^{2} - (n_x n_y -1) s_{d_{ij}}^{2}}{n_x n_y (n_x-1) (n_y-1)} ,$$ which can also alternatively be put as $$s_{d}^{2} = \frac{n_{y} s_{d_{i\cdot}}^{2}}{n_x (n_y-1)} + \frac {n_{x} s_{d_{\cdot j}}^{2}}{n_y (n_x-1)} - \frac{(n_x n_y -1) s_{d_{ij}}^{2}} {n_{x} n_{y}(n_{x}-1) (n_{y}-1)}.$$ (For differences to Cliff's (1996, p. 138) formula see notes to Row 28 ("var dij") below.) In case this calculation of $s_{d}^{2}$ yields values of less than $(1 - d^{2})/(n_x n_y -1)$, this latter formula is used for calculating the variance of delta. Column 2 contains the pooled $s_{xy}^{2}$.
[19 or ["se delta", col#]Column 2 only: metric Standard error of mean difference: $SE_{xy}=s_{xy} \sqrt{1/n_x+1/n_y}$ .
[20 or ["z/t score", col#]Column 1: z score of delta on the of the respective studdist-controlled distribution (Student's t or standard normal). Column 2: Metric z/t-score (= $\bar{d_{ij}}/SE_{xy}$). In the metric case, the t.welch decides upon assumption of equal variances for X and Y.
[21 or ["H1 tails p/CI", col#]Equals 1 for one-tailed and 2 for two-tailed testing of alternative or H_1-hypothesis, affecting CI and p values.
[22 or ["p", col#]Probability of z/t score (1-sided or 2-sided comparison as shown in row 21).
[23 or ["Cohen's d", col#]Cohen's d effect size estimate of delta. For Cliff's delta inferred from distributional non-overlap as suggested by Grissom & Kim (2005, p. 106 f.) as well as Romano, Kromrey, Coraggio, & Skowronek (2006, p. 14-15), relating to the relative positions of the distributions of X and Y. When Cliff's delta equals 0, there is no effect, and the Y and X distributions overlap completely. If there are effects, a certain percentage of non-overlap between X and Y is created, and the relative positions of the X and Y distribtions shift. The degree of non-overlap thus is a measure of effect size and is expressed as Cohen's d in terms of non-overlap between two normal distributions (based on U1 in Table 2.2.1, Cohen, 1988, p.22). See delta2cohd manual of orddom package. Column 2 returns Cohen's d assuming a pooled variance for t. See metric_t for details.
[24 or ["d CI low", col#]Column 1: Cohen's d effect size estimate of the lower boundary of confidence interval (row 15) by using the non-overlap strategy. Column 2: Confidence bands for metric Cohen's d are constructed based on the estimated standard deviation of Cohen's d's theoretical sampling distribution, assuming asymptotic normality (Hedges & Olkin, 1985), calculated as $CI_{lower/upper}= d \pm z s_d$, where $z$ is the z-score at the selected $\alpha$ level (2-tailed) of the standard normal distribution, and $$s_{d} = \sqrt{\frac{n_x + n_y}{n_x n_y} + \frac{d^2}{2(n_x + n_y)}}$$.
[25 or ["d CI high", col#]Column 1:Cohen's d estimate of upper boundary of confidence interval (row 16). Column 2: see row 24 for details.
[26 or ["var d.i", col#]Row variance of dominance/difference matrix, calculated as $(n_x-1)^{-1} \sum (d_{i\cdot}-d)^2$. The metric descriptive in column 2 is the variance of x (or $s_{x}^{2}$).
[27 or ["var dj.", col#]Column variance of dominance/difference matrix, calculated as $(n_y-1)^{-1} \sum (d_{\cdot j}-d)^2$. The metric descriptive in column 2 is the variance of y (or $s_{y}^{2}$).
[28 or ["var dij", col#]Variance of dominance/difference matrix as sample estimate according to Long et al. (2003, section 3.3 before eqn. 67):$$s_{d_{ij}}^{2}=\frac{\sum \sum(d_{ij}-d)^{2}}{n_x n_y -1}=\frac{\sum d_{ij}^{2} - \frac{(\sum d_{ij})^{2}}{n_x n_y}}{n_x n_y -1},$$ thus avoiding Cliff's original (1996, p. 138) suggestion to use $(n_x-1)(n_y-1)$ as the denominator).
[29 or ["df", col#]If the studdist parameter is not set to FALSE, column 1 returns the degrees of freedom ($df$) used for CI as well as z/t-score and z-probability estimates. In column 2 ("metric") df as used for metric t-test.
[30 or ["NNT", col#]The number needed to treat effect size (NNT, cf. Cook & Sackett, 1995) is returned based on the delta statistic as $$delta^{-1}$$ as suggested by Kraemer & Kupfer, 2006, p. 994. In column 2, the NNT is returned based on Cohen's d of the metric between-group comparison.
DEPENDENT/PAIRED GROUPS (paired argument set to TRUE) In the case of paired data (e.g. pretest-posttest comparisons of the $n_x=n_y$ same subjects), a 4-column-matrix containing 29 rows with values is returned. The ordinal statistics for $d_{ij}$ can be retrieved from the first three columns (named
within [.,1]for the $n_x=n_y$ within-pair changes (where $i=j$ in all cases);
between [.,2]for the overall distribution changes, based on all $n^2-n = n(n-1)$ comparisons where $i \ne j$, and
combined [.,3]for combined inferences $d_w + d_b$.
Here, the fourth column (named "metric") contains metric comparison data.
[1 or ["var1_X_pre", col#]Original column name of the x (or pretest) input matrix.
[2 or ["var2_Y_post", col#]Original column name of the y (or posttest) input matrix.
[3 or ["type_title", col#]Columns 1-3: Return type of the comparison, in this case "paired". Column4: In case a string header is defined by use of the comp.name argument, it is returned in column 4.
[4 or ["N #Y>X", col#]Number of occurences (#) of a posttest observation $y_i$ having a higher value than a pretest observation $x_j$: $N_{\#Y>X}=\#(y_i>x_j) ,$ limited to the respective pairs under observation in within, between or combined. Column 4 equals column 3.
[5 or ["N #Y=X", col#]Number of occurences of a posttest observation having the same value as a pretest observation, limited to the respective pairs under observation in within, between or combined. Column 4 equals column 3.
[6 or ["N #Ycol#]Number of occurences of a posttest observation having a smaller value than a pretest observation, limited to the respective pairs under observation in within, between or combined. Column 4 equals column 3.
[7 or ["PS X>Y", col#]Common Language CL effect size or Probability of Superiority (PS) of X over Y (Grissom, 1994,Grissom & Kim, 2005) (limited to the respective pairs under observation in within, between or combined): $$PS(Y>X)=\frac{\#(y_i>x_j)}{n_y \cdotp n_x} .$$. This effect size reflects the probability that a subject or case randomly chosen from the X- or pre-test-scores under observation has a higher score than than a randomly chosen case from the respective Y- or post-test-subsample (cf. Acion et al., 2006). Column 4: Assuming equal variances and population normality, the (para)metric version of the Common Language effect size is calculated as suggested by McGraw & Wong (1992, p. 363) for correlated samples by using the variance sum law to adjust the variance on the difference scores with $PS(Y>X)=\Phi (\frac{M_y-M_x}{\sqrt{s_x^{2} + s_y^{2} - 2 r_{xy} s_x s_y}})$ where $\Phi$ is the cumulative normal distribution function with $\Phi(z_{\alpha})=\alpha$.
[8 or ["PS Y>X", col#]Common Language CL effect size or Probability of Superiority (PS) of Y over X (Grissom, 1994,Grissom & Kim, 2005) (limited to the respective pairs under observation in within, between or combined). Column 4: CL (para)metric version for the correlated samples case (see row 7 above for details on calculation).
[9 or ["A X>Y", col#]Vargha and Delaney's A as stochastic superiority of X over Y, limited to the respective pairs under observation in within, between or combined. (See code{dmes} of this orddom package for details.) Column 4 equals column 3.
[10 or ["A Y>X", col#]Vargha and Delaney's A as stochastic superiority of Y over X, limited to the respective pairs under observation in within, between or combined. (See code{dmes} of this orddom package for details.) Column 4 equals column 3.
[11 or ["delta", col#]For columns 1 to 3 ("ordinal"), the respective delta for dependent groups (Cliff, 1996,Long et al., 2003,Feng, 2007) is reported. With $d_{ij}=sign(y_i-x_j)$, Column 1 reports the (within) value, which is the "difference between the proportion of individual subjects who change in one direction and the proportion of individuals who change in the other" (Cliff, 1996, p. 159), calculated as $$d_w=({\sum_{i}}_{=}\sum_{j} d_{ii})/ n ,$$ where $i=j$ in the $n = n_x = n_y$ possible paired comparisons. "The extent to which the overall distribution has moved, except for the self-comparisons" (Cliff, 1996, p. 160) is given in column 2, the delta-(between) statistic. It is estimated by the average between-subject dominance, calculated as $$d_b=({\sum_{i}}_{\not=}\sum_{j} d_{ij})/ (n (n-1)) ,$$ where $i \not= j$. Column 3 reports the combination effect $d_w + d_b$. In column 4 ("metric"), the differences between subsample means are reported: $\bar{y}-\bar{x}$.
[12 or ["1-alpha", col#]Significance or $\alpha$-level for CI estimation, given as percentage between 0 and 100.
[13 or ["CI low", col#]Confidence interval (CI) lower boundary estimate. Unless the default symmetric parameter is explicitly set to TRUE, asymmetric Confidence interval (CI) boundary estimates for ordinal differences are calculated (Feng & Cliff, 2004; Feng, 2007) as $$CI_{lower/upper}=\frac{d - d^{3} \pm t_{\alpha/2} s_{d} \sqrt{1 - 2 d^{2} + d^{4} + t_{\alpha/2}^{2} s_{d}^{2} )}}{1 - d^{2} + t_{\alpha/2}^{2} s_{d}^{2}} ,$$ with t-values at the respective significance level based on either Student's t or on z-values from the Standard Normal Distribution, depending on the studdist argument. However, using an asymmetric CI is not advisable when the combined delta estimate (column 3) is to be used for inferences. An asymmetric CI may also reduce power of $d_b$ value given in [10,2], especially in small paired samples (Cliff, 1996, p. 165). To obtain symmetric CI estimates with $CI_{lower/upper}=d \pm t_{\alpha/2} s_{d}$, the default symmetric argument must be set to TRUE. In any case, if $d = \pm 1$, one CI is set as equal to d, the other is calculated as $$CI_{lower/upper}=((n_{b}-t_{\alpha/2}^{2}))(n_{b}+t_{\alpha/2}^{2})^{-1},$$ where $t_{\alpha/2}$ is the t-value or z-score at the selected $\alpha$ level (1- or 2-tailed) of the respective studdist-controlled distribution, and $n_{b}$ the number of observations or cases in the smaller of the two samples.
[14 or ["CI high", col#]Confidence interval upper boundary estimate (see row 13).
[15 or ["s delta", col#]Estimated standard deviation of the respective delta statistic. Column 4 reports the metric standard deviation of the paired (within) differences.
[16 or ["var delta", col#]Unbiased estimates of the variances of the respective delta statistic. Column 1 reports the within value, calculated as $$s_{d_w}^{2}=(n(n-1))^{-1}(\sum (d_{ii}-d_w)^{2}).$$ Please note that in various pieces of the available research literature (e.g. Cliff, 1996, eq. 6.8, p. 161), $s_{d_w}^{2}$ is erroneously reported to be calculated as $s_{d_w}^{2}=(n-1)^{-1}(\sum (d_{ii}-d_w)^{2}).$ The denominator, however must read $n(n-1)$ as "using just $(n-1)$ would give the variance of the individual $d_{ii}$ whereas we want the variance of $d_w$, which is a kind of mean" (Feng, 07.02.2011, personal communication). The (between) unbiased estimate in column 2 is calculated as $s_{d_b}^{2}=[(n-1)^{2}(\sum (d_{i \cdot}-d_b)^{2} + \sum (d_{\cdot j}-d_b)^{2} + 2 \sum (d_{i \cdot}-d_b)(d_{\cdot j}-d_b)) - \sum \sum (d_{ij}-d_b)^{2} - \sum \sum (d_{ij}-d_b)(d_{ji}-d_b)][n(n-1)(n-2)(n-3)]^{-1} .$ In case this formula renderes negative variance estimates for $s_{db}^{2}$ estimates by use of this formula, the between variance is alternatively calculated as $s_{db}^{2}=(1-d_b^{2})/(n^{2}-n-1)$ (see Long et al. (2003, par after eqn. 66) for a related discussion). Since $d_w$ and $d_b$ are interdependent, the combined effect involves taking into account their estimated covariance when calculating the unbiased estimate for the variance for the sum of $d_w$ and $d_b$, which is reported in column 3 as $s_{d_w+d_b}^{2}=s_{d_w}^{2} + s_{d_b}^{2} + 2 \widehat{cov}(d_b,d_w),$ with $\widehat{cov}(d_b,d_w)=$ $\left( \sum_{i} [d_{ii}(\sum_{j}(d_{ij}) + \sum_{j}(d_{ji})] - 2n(n-1) d_b d_w \right) (n(n-1)(n-2))^{-1}$. Column 4 reports the metric variance of the paired (within) differences.
[17 or ["z/t score", col#]z score of delta. In column 4 ("metric") equal to the t-test score (assuming equal variances).
[18 or ["H1 tails p/CI", col#]Equals 1 for one-tailed and 2 for two-tailed testing of alternative or H_1-hypothesis, affecting CI and p values.
[19 or ["p", col#]Probability of z-score (1 or 2-tailed comparison as shown in row 18).
[20 or ["Cohen's d", col#]Cohen's d estimate of the respective delta value (see above). In the metric case, the between group t-value and the original standard deviations are also used for the paired case to avoid overestimation of the effect size (Dunlap et al., 1996). See delta2cohd for details. Not available for the combined delta in column 3.
[21 or ["d CI low", col#]Column 1 and 2: Cohen's d estimate of lower boundary of the respective confidence interval (row 13) by using the non-overlap calculation strategy. Column 3: Not available. Column 4: Confidence bands for metric Cohen's d are constructed based on the estimated standard deviation of Cohen's d's theoretical sampling distribution, assuming asymptotic normality (Hedges & Olkin, 1985), calculated as $CI_{lower/upper}= d \pm z s_d$, where $z$ is the z-score at the selected $\alpha$ level (2-tailed) of the standard normal distribution, and $$s_{d} = \sqrt{\frac{n_x + n_y}{n_x n_y} + \frac{d^2}{2(n_x + n_y)}}$$.
[22 or ["d CI high", col#]Cohen's d estimate of upper boundary of the respective confidence interval (see row 21 for calculation details).
[23,3] or ["var d.i",combined]Component of $s_{d_w+d_b}^2$: $s_{di.}^2$ (Available for the combined analyses in column 3 only.) The metric descriptive in column 4 is the variance of x (or $s_{x}^{2}$.
[24,3] or ["var dj.",combined]Component of $s_{d_w+d_b}^2$: $s_{d.i}^2$ (Third column only.) The metric descriptive in column 4 is the variance of y (or $s_{y}^{2}$.
[25,3] or ["cov(di,dj)",combined]Component of $s_{d_w+d_b}^2$: $cov(d_{i.},d_{.j})$ (Third column only.)
[26,3] or ["var dij",combined]Component of $s_{d_w+d_b}^2$: $s_{d_{ij}}^2$ (Third column only.)
[27,3] or ["cov(dih,dhi)",combined]Component of $s_{d_w+d_b}^2$: $cov(d_{ih},d_{hi})$ (Third column only.)
[28,3] or ["cov(db,dw)",combined]Estimated covariance between $d_b$ and $d_w$: $\widehat{cov}(d_b,d_w)$ (for purposes of combined inferences). (Third column only.)
[29 or ["df", col#]Unless the studdist argument is not set to FALSE, the degrees of Freedom df used for the CI and z-score calculations are reported in column 1. Column 2 returns the df used for the metric t-test for dependent samples.
[30 or ["NNT", col#]In column 1 and 2, the number needed to treat effect size (NNT, cf. Cook & Sackett, 1995) are returned, based on the underlying delta statistics with NNT=$$delta^{-1}$$ as suggested by Kraemer & Kupfer, 2006, p. 994. (Column 3 is empty.). In column 4, the NNT is returned based on Cohen's d of the metric comparison.

References

Acion, L., Peterson, J.J., Temple, S., & Arndt, S. (2006). Probabilistic index: an intuitive non-parametric approach to measuring the size of treatment effects. Statistics in Medicine, 25, 591 - 602. Cliff, N. (1996). Ordinal Methods for Behavioral Data Analysis. Mahwah, NJ: Lawrence Erlbaum. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd edition). New York: Academic Press. Cook, R.J. & Sackett, D.L. (1995). The number needed to treat: A clinically useful measure of treatment effect. British Medical Journal, 310, 452 - 454. Dunlap, W. P., Cortina, J. M., Vaslow, J. B., & Burke, M. J. (1996). Meta-analysis of experiments with matched groups or repeated measures designs. Psychological Methods, 1, 170 - 177. Feng, D. (2007). Robustness and Power of Ordinal d for Paired Data. In Shlomo S. Sawilowsky (Ed.), Real Data Analysis (pp. 163-183). Greenwich, CT : Information Age Publishing. Feng, D., & Cliff, N. (2004). Monte Carlo Evaluation of Ordinal d with Improved Confidence Interval. Journal of Modern Applied Statistical Methods, 3(2), 322-332. Long, J. D., Feng, D., & Cliff, N. (2003). Ordinal analysis of behavioral data. In J. Schinka & W. F. Velicer (eds.), Research Methods in Psychology. Volume 2 of Handbook of Psychology (I. B. Weiner, Editor-in-Chief). New York: John Wiley & Sons. Grissom, R.J. (1994). Probability of the superior outcome of one treatment over another. Journal of Applied Psychology, 79, 314-316. Grissom, R.J. & Kim, J.J. (2005). Effect sizes for research. A broad practical approach. Mahwah, NJ, USA: Erlbaum. Hedges, L.V. & Olkin, I. (1985). Statistical methods for meta-analysis. San Diego, CA, USA: Academic Press. Kraemer, H.C. & Kupfer, D.J. (2006). Size of Treatment Effects and Their Importance to Clinical Research and Practice. Biological Psychiatry, 59, 990-996. McGraw, K.O. & Wong, S.P. (1992). A common language effect size statistic. Psychological Bulletin, 111, 361-365. Romano, J., Kromrey, J. D., Coraggio, J., & Skowronek, J. (2006). Appropriate statistics for ordinal level data: Should we really be using t-test and Cohen's d for evaluating group differences on the NSSE and other surveys? Paper presented at the annual meeting of the Florida Association of Institutional Research, Feb. 1-3, 2006, Cocoa Beach, Florida. Last retrieved January 2, 2012 from www.florida-air.org/romano06.pdf

Examples

Run this code

#Independent Samples (Data taken from Long et al. (2003), Table 3
x<-t(matrix(c(3,3,3,4,5,6,12,12,13,14,15,15,15,15,15,16,18,18,18,23,23,27,28,28,43),1))
colnames(x)<-c("Nonalcohol.")
y<-t(matrix(c(1,4,6,7,7,14,14,18,19,20,21,24,25,26,26,26,27,28,28,30,33,33,44,45,50),1))
colnames(y)<-c("Alcoholic")
orddom(x,y,paired=FALSE,outputfile="tmp_r.txt")
#Paired Comparison with data written to file (Data taken from Long et al. (2003), Table 4
x<-t(matrix(c(2,6,6,7,7,8,8,9,9,9,10,10,10,11,11,12,13,14,15,16),1))
colnames(x)<-c("Incidental")
y<-t(matrix(c(4,11,8,9,10,11,11,5,14,12,13,10,14,16,14,13,15,15,16,10),1))
colnames(y)<-c("Intentional")
orddom_f(y,x,paired=TRUE,symmetric=FALSE)
#Directly returns d_b of the paired comparison
orddom(x,y,,TRUE,,,)[11,2]