Generates simple list of nonparametric ordinal effect size measures such as
-the Probability of Superiority (or discrete case Common Language) effect size,
-the Vargha and Delaney's A (or area under the receiver operating characteristic curve, AUC)
-Cliff's delta (or success rate difference, SRD), and
-the number needed to treat (NNT) effect size (based on Cliff's delta value).
Usage
dmes(x,y)
Arguments
x
A vector or 1 column matrix with $n_x$ values from (control or pre-test or comparison) group X
y
A vector or 1 column matrix with $n_y$ values from (treatment or post-test) group Y
Value
$nx
Vector or sample size of x, $n_x$.
$ny
Vector or sample size of y, $n_y$
$PSc
Discrete case Common Language CL effect size or Probability of Superiority (PS) of all values of Y over all values of X: $$PS_c(Y>X)=\frac{\#(y_i>x_j)}{n_y n_x}$$,
where $i=\{1, 2, ... , n_y\}$ and $j=\{1, 2, ... , n_x\}$. See orddomPS Y>X for details.)
$Ac
Vargha & Delaney's A or Area under the receiver operating characteristics curve (AUC) for all possible comparisons: $$A(Y>X)=[\#(y_i>x_j) + .5(\#(y_i=x_j)] (n_y n_x)^{-1}$$,
where $i=\{1, 2, ... , n_y\}$ and $j=\{1, 2, ... , n_x\}$. See orddomA Y>X for details.)
$dc
Success rate difference when comparing all values of Y with all values of X: $$d_c(Y>X)=\frac{\#(y_i>x_j)-\#(y_iorddom Cliff's delta for independent groups for details.
Note that in the paired samples case with $n_y=n_x$, $dc does not return the combined estimate, i.e. $$dc<>dw+db$!
$NNTc
Number needed to treat, based on the success rate difference or $\$dc^{-1}$. See orddom "NNT" for details.
$PSw
When sample sizes are equal, this value returns the Probability of Superiority (PS) for within-changes, i.e. alle paired values: $PS(Y>X)=\#(y_i>x_i)/(n_y n_x)$, limited to the $n_x=n_y$ paired cases where $i=\{1,2,...,n_x=n_y\}$. (For unequal sample sizes, this equals $PSc.)
$Aw
When sample sizes are equal, this value returns A for the paired subsample values, i.e. limited to the $n_x=n_y$ paired cases where $i=j=\{1,2,...,n_x=n_y\}$. (For unequal sample sizes, this equals $Ac.)
$dw
When $n_x=n_y$, this value returns Cliff's delta-within, i.e. paired comparisons limited to the diagonal of the dominance matrix or those cases where $i=j$. (For unequal sample sizes, this equals $dc.)
$NNTw
Number needed to treat, based on the within-case-success rate difference or $\$dw^{-1}$. See orddomNNT within for dependent groups for details.
$PSb
When sample sizes are equal, this gives the Probability of Superiority (PS) for all cases but within-pair changes, i.e.: $$PS_b(Y>X)=\frac{\#(y_i>x_j)}{n_y n_x}$$,
limited to those cases where $i<>j$. (For unequal sample sizes, this equals $PSc and $PSw.)
$Ab
When sample sizes are equal, this value returns A for all cases where $i<>j$. (For unequal sample sizes, this equals $Ac.)
$db
When $n_x=n_y$, this value returns Cliff's delta-between, i.e. all but the paired comparisons or excepting the diagonal of the dominance matrix. The parameter is calculated by taking only those ordinal comparisons into account where $i<>j$. (For unequal sample sizes, this equals $dc.)
$NNTb
Number needed to treat, based on Cliff's delta-between or $\$db^{-1}$. See orddomNNT between for dependent groups for details.
Details
Based on the dominance matrix created by direct ordinal comparison of values of Y with values of X, an associative list is returned.
References
Delaney, H.D. & Vargha, A. (2002). Comparing Several Robust Tests of Stochastic Equality With Ordinally Scaled Variables and Small to Moderate Sized Samples. Psychological Methods, 7, 485-503.
Kraemer, H.C. & Kupfer, D.J. (2006). Size of Treatment Effects and Their Importance to Clinical Research and Practice. Biological Psychiatry, 59, 990-996.
Ruscio, J. & Mullen, T. (2012). Confidence Intervals for the Probability of Superiority Effect Size Measure and the Area Under a Receiver Operating Characteristic Curve. Multivariate Behavioral Research, 47, 221-223.
Vargha, A., & Delaney, H. D. (1998). The Kruskal-Wallis test and stochastic homogeneity. Journal of Educational and Behavioral Statistics, 23, 170-192.
Vargha, A., & Delaney, H. D. (2000). A critique and improvement of the CL common language effect size statistic of McGraw and Wong. Journal of Educational and Behavioral Statistics, 25, 101-132.