HHG (version 2.3.7)

hhg.univariate.ks.nulltable: The K-sample test null tables for all partition sizes

Description

Functions for creating null table objects, for the omnibus distribution-free test of equality of distributions among K groups, as described in Heller et al. (2016). To be used for the p-value computation, see examples in hhg.univariate.ks.pvalue.

Usage

hhg.univariate.ks.nulltable(group.sizes,mmin=2,
mmax=max(4,round(min(group.sizes)/3)),variant = 'KSample-Variant',
aggregation.type='sum',score.type='LikelihoodRatio',
nr.replicates=1000,keep.simulation.data=F,
nr.atoms = nr_bins_equipartition(sum(group.sizes)),
compress=F,compress.p0=0.001,compress.p=0.99,compress.p1=0.000001)

Value

m.stats

If keep.simulation.data= TRUE, a matrix with nr.replicates rows and mmax-mmin+1 columns of null test statistics.

univariate.object

A useful format of the null tables for computing p-values efficiently.

Arguments

group.sizes

the number of observations in each group.

mmin

The minimum partition size of the ranked observations, default value is 2.

mmax

The maximum partition size of the ranked observations, default value is 1/3 the number of observations in the smallest group.

variant

Default value is 'KSample-Variant'. Setting the variant to 'KSample-Equipartition' performs the K-sample tests over partitions of the data where splits between cells are at least \(n/nr.atoms\) apart.

aggregation.type

a character string specifying the aggregation type, must be one of "sum" (default) or "max".

score.type

a character string specifying the score type, must be one of "LikelihoodRatio" (default) or "Pearson".

nr.replicates

The number of permutations for the null distribution.

keep.simulation.data

TRUE/FALSE. If TRUE, then in addition to the sorted statistics per column, the original matrix of size nr.replicates by mmax-mmin+1 is also stored.

nr.atoms

For variant=='KSample-Equipartition' type tests, sets the number of possible split points in the data. The default value is the minimum between \(n\) and \(60+0.5*\sqrt{n}\).

compress

TRUE or FALSE. If enabled, null tables are compressed: The lower compress.p part of the null statistics is kept at a compress.p0 resolution, while the upper part is kept at a compress.p1 resolution (which is finer).

.

compress.p0

Parameter for compression. This is the resolution for the lower compress.p part of the null distribution.

compress.p

Parameter for compression. Part of the null distribution to compress.

compress.p1

Parameter for compression. This is the resolution for the upper value of the null distribution.

Author

Barak Brill and Shachar Kaufman.

Details

In order to compute the null distributions for a test statistic (with a specific aggregation and score type, and all partition sizes), the only necessary information is the group sizes (the test statistic is "distribution free"). The accuracy of the quantiles of the null distribution depend on the number of replicates used for constructing the null tables. The necessary accuracy depends on the threshold used for rejection of the null hypotheses.

This function creates an object for efficiently storing the null distribution of the test statistics (by partition size m). Use the returned object, together with hhg.univariate.ks.pvalue to compute the P-value for the statistics computed by hhg.univariate.ks.stat

Generated null tables also hold the distribution of statistics for combination types (comb.type=='MinP' and comb.type=='Fisher'), used by hhg.univariate.ks.combined.test.

Variant type "KSample-Equipartition" is the computationally efficient version of the K-sample test. calculation time is reducing by aggregating over a subset of partitions, where a split between cells may be performed only every \(n/nr.atoms\) observations. This allows for a complexity of O(nr.atoms^2) (instead of O(n^2)). Computationly efficient versions are available for aggregation.type=='sum' and aggregation.type=='max' variants.

Null tables may be compressed, using the compress argument. For each of the partition sizes (i.e. m or mXm), the null distribution is held at a compress.p0 resolution up to the compress.p percentile. Beyond that value, the distribution is held at a finer resolution defined by compress.p1 (since higher values are attained when a relation exists in the data, this is required for computing the p-value accurately.)

References

Heller, R., Heller, Y., Kaufman S., Brill B, & Gorfine, M. (2016). Consistent Distribution-Free K-Sample and Independence Tests for Univariate Random Variables, JMLR 17(29):1-54 https://www.jmlr.org/papers/volume17/14-441/14-441.pdf

Brill B. (2016) Scalable Non-Parametric Tests of Independence (master's thesis) https://tau.userservices.exlibrisgroup.com/discovery/delivery/972TAU_INST:TAU/12397000130004146?lang=he&viewerServiceCode=AlmaViewer

Examples

if (FALSE) {
#Testing for the difference between two groups, each from a normal mixture:
N0=30
N1=30

#null table for aggregation by summation: 
sum.nulltable = hhg.univariate.ks.nulltable(c(N0,N1), nr.replicates=100)
#default nr. of replicates is 1000,
#but may take several seconds. For illustration only, we use 100 replicates,
#but it is highly recommended to use at least 1000 in practice. 

#null table for aggregation by maximization: 
max.nulltable = hhg.univariate.ks.nulltable(c(N0,N1), aggregation.type = 'max',
  score.type='LikelihoodRatio', mmin = 3, mmax = 5, nr.replicates = 100)
#default nr. of replicates is 1000, but may take several seconds. For illustration only,
#we use 100 replicates, but it is highly recommended to use at least 1000 in practice.

#null tables for aggregation by summation and maximization, for large data variants:
#make sure to change mmax, such that mmax<= nr.atoms

N0_large = 5000
N1_large = 5000

Sm.EQP.null.table = hhg.univariate.ks.nulltable(c(N0_large,N1_large),
nr.replicates=200, variant = 'KSample-Equipartition', mmax = 30)
Mm.EQP.null.table = hhg.univariate.ks.nulltable(c(N0_large,N1_large),
nr.replicates=200, aggregation.type='max', variant = 'KSample-Equipartition', mmax = 30)
}