Learn R Programming
MXM (version 0.8.7)

CondIndTests: MXM Conditional Independence Tests

Description

Currently the MXM package supports numerous tests for different types of target (dependent) and predictor (independent) variables. The target variable can be of continuous, discrete, categorical and of survival type. As for the predictor variables, they can be continuous, categorical or mixed. The testIndFisher and the gSquare tests have two things in common. They do not use a model implicitly (i.e. estimate some beta coefficients), even though there is an underlying assumed one. Secondly they are pure tests of independence (again, with assumptions required). As for the other tests, they share one thing in common. For all of them, two parametric models must be fit. The null model containing the conditioning set of variables alone and the alternative model containing the conditioning set and the candidate variable. The significance of the new variable is assessed via a log-likelihood ratio test with the appropriate degrees of freedom. All of these tests are summarized in the below table. llll{ Target variable Predictor variables Available tests Short explanation Continuous Continuous testIndFisher (robust) Partial correlation Continuous Continuous testIndSpearman Partial correlation Continuous Mixed testIndReg (robust) Linear regression Continuous Mixed testIndRQ Median regression Proportions Continuous testIndFisher (robust) Partial correlation after logit transformation Proportions Continuous testIndSpearman Partial correlation after logit transformation Proportions Mixed testIndReg(robust) Linear regression after logit transformation Proportions Mixed testIndRQ Median regression after logit transformation Proportions Mixed testIndBeta Beta regression Discrete Mixed testIndPois Poisson regression Discrete Mixed testIndZIP Zero Inflated Poisson regression Numerical Mixed testIndNB Negative binomial regression Factor with two Mixed testIndLogistic Binary logistic regression levels or binary Factor with more Mixed testIndLogistic Multinomial logistic regression than two levels (unordered) Factor with more than Mixed testIndLogistic Ordinal logistic regression two levels (ordered) Categorical Categorical gSquare G-squared test of independence Categorical Categorical testIndLogistic Multinomial logistic regression Categorical Categorical testIndLogistic Ordinal logistic regression Continuous, proportions, Mixed testIndSpeedglm Linear, binary logistic binary or counts or poisson gression Survival Mixed censIndCR Cox regression Survival Mixed censIndWR Weibull regression Case-control Mixed testIndClogit Conditional logistic regression Multivariate continuous Mixed testIndMVreg Multivariate linear regression Compositional data Mixed testIndMVreg Multivariate linear regression (no zeros) after multivariate logit transformation Longitudinal Mixed TestIndGLMM (Generalised) linear mixed models }

Arguments

Tests

  1. testIndFisher. This is a standard test of independence when both the target and the set of predictor variables are continuous (continuous-continuous). When the joint multivariate normality of all the variables is assumed, we know that if a correlation is zero this means that the two variables are independent. Moving in this spirit, when the partial correlation between the target variable and the new predictor variable conditioning on a set of (predictor) variables is zero, then we have evidence to say they are independent as well. An easy way to calculate the partial correlation between the target and a predictor variable conditioning on some other variables is to regress the both the target and the new variable on the conditioning set. The correlation coefficient of the residuals produced by the two regressions equals the partial correlation coefficient. If the robust option is selected, the two aforementioned regression models are fitted using M estimators (Marona et al., 2006). If the target variable consists of proportions or percentages (within the (0, 1) interval), the logit transformation is applied beforehand.
  2. testIndSpearman. This is a non-parametric alternative totestIndFishertest. It is a bit slower than its competitor, yet very fast and suggested when normality assumption breaks down or outliers are present. In fact, within SES, what happens is that the ranks of the target and of the dataset (predictor variables) are computed and thetestIndSpearmanis aplied. This is faster than applying Fisher with M estimators as described above. If the target variable consists of proportions or percentages (within the (0, 1) interval), the logit transformation is applied beforehand.
  3. testIndReg. In the case of target-predictors being continuous-mixed or continuous-categorical, the suggested test is via the standard linear regression. In this case, two linear regression models are fitted. One with the conditioning set only and one with the conditioning set plus the new variable. The significance of the new variable is assessed via the F test, which calculates the residual sum of squares of the two models. The reason for the F test is because the new variable may be categorical and in this case the t test cannot be used. It makes sense to say, that this test can be used instead of thetestIndFisher, but it will be slower. If the robust option is selected, the two models are fitted using M estimators (Marona et al. 2006). If the target variable consists of proportions or percentages (within the (0, 1) interval), the logit transformation is applied beforehand.
  4. testIndRQ. An alternative totestIndRegfor the case of continuous-mixed (or continuous-continuous) variables is thetestIndRQ. Instead of fitting two linear regression models, which model the expected value, one can choose to model the median of the distribution (Koenker, 2005). The significance of the new variable is assessed via a rank based test calibrated with an F distribution (Gutenbrunner et al., 1993). The reason for this is that we performed simulation studies and saw that this type of test attains the type I error in contrast to the log-likelihood ratio test. The benefit of this regression is that it is robust, in contrast to the classical linear regression. If the target variable consists of proportions or percentages (within the (0, 1) interval), the logit transformation is applied beforehand.
  5. testIndBeta. When the target is proportion (or percentage, i.e., between 0 and 1, not inclusive) the user can fit a regression model assuming a beta distribution. The predictor variables can be either conitnuous, categorical or mixed. The procedure is the same as in thetestIndRegcase.
  6. Alternatives to testIndBeta. Instead oftestIndBetathe user has the option to choose all the previous to that mentioned tests by transforming the target variable with the logit transformation. In this way, the support of the target becomes the whole of R^d and then depending on the type of the predictors and whether a robust approach is required or not, there is a variety of alternative to beta regression tests.
  7. testIndPois. When the target is discrete, and in specific count data, the default test is via the Poisson regression. The predictor variables can be either conitnuous, categorical or mixed. The procedure is the same as in all the previously regression model based tests.
  8. testIndNB. As an alternative to the Poisson regression, we have included the Negative binomial regression to capture cases of overdispersion. The predictor variables can be either conitnuous, categorical or mixed.
  9. testIndZIP. When the number of zeros is more than expected under a Poisson model, the zero inflated poisson regression is to be employed. The predictor variables can be either conitnuous, categorical or mixed.
  10. testIndLogistic(Binomial). When the target is categorical with only two outcomes, success or failure for example, then a binary logistic regression is to be used. Whether regression or classification is the task of interest, this method is applicable. The advantage of this over a linear or quadratic discriminant analysis is that it allows for categorical predictor variables as well and for mixed types of predictors.
  11. testIndLogistic(Un-ordered multinomial). If the target has more than two outcomes, but it is of nominal type, there is no ordering of the outcomes, multinomial logistic regression will be employed. Again, this regression is suitable for classification purposes as well and it to allows for categorical predictor variables.
  12. testIndLogistic(Ordered multinomial). This is a special case of multinomial regression, in which case the outcomes have an ordering, such asnot satisfied,neutral,satisfied. The appropriate method is ordinal logistic regression.
  13. testIndSpeedglm. If you have a few tens of thousands of observations, the default functions for linear, binary logistic and poisson regression will be slow causing the computer to jamm. For this reason, memory efficient handling regressions should be used.
  14. gSquare. If all variables, both the target and predictors are categorical the default test is the G-square test of independence. It is similar to the chi-squared test of independence, but instead of using the chi-squared metric between the observed and estimated frequencies in contingency tables, the Kullback-Leibler divergence of the observed from the estimated frequencies is used. The asymptotic distribution of the test statistic is a chi-squared distribution on some appropriate degrees of freedom. The target variable can be either ordered or unordered with two or more outcomes.
  15. Alternatives to gSquare. An alternative to thegSquaretest is thetestIndLogistic. Depending on the nature of the target, binary, un-ordered multinomial or ordered multinomial the appropriate regression model is fitted.
  16. censIndCR. For the case of time-to-event data, a Cox regression model is employed. The predictor variables can be either continuous, categorical or mixed. Again, the log-likelihood ratio test is used to assess the significance of the new variable.
  17. censIndWR. A second model for the case of time-to-event data, a Weibull regression model is employed. The predictor variables can be either continuous, categorical or mixed. Again, the log-likelihood ratio test is used to assess the significance of the new variable. Unlike the semi-parametric Cox model, the Weibull model is fully parametric.
  18. testIndClogit. When the data come from a case-control study, the suitable test is via conditional logistic regression.
  19. testIndMVReg. In the case of multivariate continuous targets, the suggested test is via a multivariate linear regression. The target variable can be compositional data as well. These are positive data, whose vectors sum to 1. They can sum to any constant, as long as it the same, but for convenience reasons we assume that they are normalised to sum to 1. In this case the additive log-ratio transformation (multivariate logit transformation) is applied beforehand.
  20. testIndGLMM. In the case of a longitudinal or clustered targets (continuous, proportions, binary or counts), the suggested test is via a (generalised) linear mixed model.

Details

These tests can be called by SES or individually by the user.

References

Aitchison J. (1986). The Statistical Analysis of Compositional Data, Chapman & Hall; reprinted in 2003, with additional material, by The Blackburn Press. Brown P.J. (1994). Measurement, Regression and Calibration. Oxford Science Publications. Cox D.R. (1972). Regression models and life-tables. J. R. Stat. Soc., 34, 187-220. Draper, N.R. and Smith H. (1988). Applied regression analysis. New York, Wiley, 3rd edition. Fieller E.C. and Pearson E.S. (1961). Tests for rank correlation coefficients: II. Biometrika, 48(1 & 2): 29-40. Ferrari S.L.P. and Cribari-Neto F. (2004). Beta Regression for Modelling Rates and Proportions. Journal of Applied Statistics, 31(7): 799-815. Gutenbrunner C., Jureckova J., Koenker R. and Portnoy S. (1993). Tests of Linear Hypothesis based on Regression Rank Scores, Journal of NonParametric Statistics 2, 307-331. Hoerl A.E. and Kennard R.W. (1970). Ridge regression: Biased estimation for nonorthogonal problems. Technometrics, 12(1): 55-67. Joseph M.H. (2011). Negative Binomial Regression. Cambridge University Press, 2nd edition. Koenker R.W. (2005). Quantile Regression. Cambridge University Press. Lagani V., Kortas G. and Tsamardinos I. (2013). Biomarker signature identification in "omics" with multiclass outcome. Computational and Structural Biotechnology Journal, 6(7): 1-7. Lagani V. and Tsamardinos I. (2010). Structure-based variable selection for survival data. Bioinformatics Journal 16(15): 1887-1894. Lambert D. (1992). Zero-inflated Poisson regression, with an application to defects in manufacturing. Technometrics 34(1)1: 1-14. Mardia K.V., Kent J.T. and Bibby J.M. (1979). Multivariate Analysis. Academic Press, New York, USA. Maronna R.D. Yohai M.V. (2006). Robust Statistics, Theory and Methods. Wiley. McCullagh P. and Nelder J.A. (1989). Generalized linear models. CRC press, USA, 2nd edition. Pinheiro J., and D. Bates. Mixed-effects models in S and S-PLUS. Springer Science & Business Media, 2006. Spirtes P., Glymour C. and Scheines R. (2001). Causation, Prediction, and Search. The MIT Press, Cambridge, MA, USA, 3nd edition. Tsamardinos I., Lagani V. and Pappas D. (2012). Discovering multiple, equivalent biomarker signatures. In proceedings of the 7th conference of the Hellenic Society for Computational Biology & Bioinformatics - HSCBB12.