MXM (version 0.7)

testIndZIP: Zero inflated Poisson regression conditional independence test for discrete (counts) class dependent variables

Description

The main task of this test is to provide a p-value PVALUE for the null hypothesis: feature 'X' is independent from 'TARGET' given a conditioning set CS. The pvalue is calculated by comparing a zero inflated Poisson regression model based on the conditioning set CS against a model whose regressor are both X and CS. The comparison is performed through a chi-square test with the appropriate degrees of freedom on the difference between the deviances of the two models.

Usage

testIndZIP(target, dataset, xIndex, csIndex, dataInfo = NULL, univariateModels = NULL, 
hash = FALSE, stat_hash = NULL, pvalue_hash = NULL, robust = FALSE)

Arguments

target

A numeric vector containing the values of the target variable.

dataset

A numeric matrix or data frame, in case of categorical predictors (factors), containing the variables for performing the test. Rows as samples and columns as features.

xIndex

The index of the variable whose association with the target we want to test.

csIndex

The indices of the variables to condition on.

dataInfo

list object with information on the structure of the data. Default value is NULL.

univariateModels

Fast alternative to the hash object for univariate test. List with vectors "pvalues" (p-values), "stats" (statistics) and "flags" (flag = TRUE if the test was succesful) representing the univariate association of each variable with the target. Default val

hash

A boolean variable which indicates whether (TRUE) or not (FALSE) to use tha hash-based implementation of the statistics of SES. Default value is FALSE. If TRUE you have to specify the stat_hash argument and the pvalue_hash argument.

stat_hash

A hash object (hash package required) which contains the cached generated statistics of a SES run in the current dataset, using the current test.

pvalue_hash

A hash object (hash package required) which contains the cached generated p-values of a SES run in the current dataset, using the current test.

robust

A boolean variable which indicates whether (TRUE) or not (FALSE) to use a robust version of the statistical test if it is available. It takes more time than non robust version but it is suggested in case of outliers. Default value is FALSE. This test does

Value

• A list including:
• pvalue
  A numeric value that represents the generated p-value due to zero inflated Poisson regression (see reference below).
• stat
  A numeric value that represents the generated statistic due to zero inflated Poisson regression(see reference below).
• flag
  A numeric value (control flag) which indicates whether the test was succesful (0) or not (1).
• stat_hash
  The current hash object used for the statistics. See argument stat_hash and details. If argument hash = FALSE this is NULL.
• pvalue_hash
  The current hash object used for the p-values. See argument stat_hash and details. If argument hash = FALSE this is NULL.

Details

If hash = TRUE, testIndZIP requires the arguments 'stat_hash' and 'pvalue_hash' for the hash-based implementation of the statistic test. These hash Objects are produced or updated by each run of SES (if hash == TRUE) and they can be reused in order to speed up next runs of the current statistic test. If "SESoutput" is the output of a SES run, then these objects can be retrieved by SESoutput@hashObject$stat_hash and the SESoutput@hashObject$pvalue_hash. If you have more zeros than expected under a Poisson model, not overdispersion, then zero inlfated Poisson is to be used. Important: Use these arguments only with the same dataset that was used at initialization. For all the available conditional independence tests that are currently included on the package, please see "?CondIndTests".

References

Lambert, Diane (1992). Zero-inflated Poisson regression, with an application to defects in manufacturing. Technometrics 34(1)1: 1-14. McCullagh, Peter, and John A. Nelder. Generalized linear models. CRC press, USA, 2nd edition, 1989.

See Also

SES, testIndPois, testIndNB, gSquare, CondIndTests

Examples

#simulate a dataset with continuous data
dataset <- matrix( runif(50 * 200, 1, 50), ncol = 50 ) 
#the target feature is the last column of the dataset as a vector
target <- rpois(200, 10)
target[sample(1:100, 30)] <- 0
results <- testIndZIP(target, dataset, xIndex = 14, csIndex = 10)
results

#require(gRbase)  #for faster computations in the internal functions
#run the SES algorithm using the testIndZIP conditional independence test
sesObject <- SES(target, dataset, max_k = 3, threshold = 0.05, test = "testIndZIP");
sesObject2 <- SES(target, dataset, max_k = 3, threshold = 0.05, test = "testIndPois");
#print summary of the SES output
summary(sesObject);
summary(sesObject2);
#plot the SES output
plot(sesObject, mode = "all");