valid: Compute validation criterion for PLS, sPLS, PLS-DA and sPLS-DA

Description

Function to estimate measures of the prediction error for fitted PLS, sparse PLS, PLS-DA and sparse PLS-DA models. M-fold and leave-one-out cross-validation are implemented.

Usage

## S3 method for class 'pls':
valid(object, validation = c("Mfold", "loo"), folds = 10,
      max.iter = 500, tol = 1e-06, ...)	

## S3 method for class 'spls':
valid(object,  validation = c("Mfold", "loo"), folds = 10,
         max.iter = 500, tol = 1e-06, ...)

## S3 method for class 'plsda':
valid(object, method = c("all", "max.dist", "centroids.dist", 
                                         "mahalanobis.dist"),
         validation = c("Mfold", "loo"), folds = 10,
         max.iter = 500, tol = 1e-06, ...)	

## S3 method for class 'splsda':
valid(object, method = c("all", "max.dist", "centroids.dist", 
                                          "mahalanobis.dist"),
         validation = c("Mfold", "loo"), folds = 10,
         max.iter = 500, tol = 1e-06, ...)

Arguments

object

object of class inheriting from "pls", "plsda", "spls" or "splsda".

method

prediction method to be applied for plsda or splsda. Should be a subset of "max.dist", "centroids.dist", "mahalanobis.dist". Default is "all". See

validation

character. What kind of (internal) validation to use, matching one of "Mfold" or "loo" (see below). Default is "Mfold".

folds

the folds in the Mfold cross-validation. See Details.

max.iter

integer, the maximum number of iterations.

tol

a not negative real, the tolerance used in the iterative algorithm.

...

arguments to pass to nearZeroVar.

Value

For PLS and sPLS models, valid produces a list with the following components:
MSEPMean Square Error Prediction for each $Y$ variable.
R2a matrix of $R^2$ values of the $Y$-variables for models with $1, \ldots ,$ncomp components.
Q2if $Y$ containts one variable, a vector of $Q^2$ values else a list with a matrix of $Q^2$ values for each $Y$-variable. Note that in the specific case of an sPLS model, it is better to have a look at the Q2.total criterion.
Q2.totala vector of $Q^2$-total values for models with $1, \ldots ,$ncomp components.
For PLS-DA and sPLS-DA models, valid produces a matrix of classification error rate estimation. The dimensions correspond to the components in the model and to the prediction method used, respectively.

encoding

latin1

Details

For fitted PLS and sPLS regression models, valid estimates the mean squared error of prediction (MSEP), $R^2$, and $Q^2$ to assess the predictive validity of the model using M-fold or leave-one-out cross-validation. Note that only the classic, regression and invariant modes can be applied. If validation = "Mfold", M-fold cross-validation is performed. How many folds to generate is selected by specifying the number of folds in folds. The folds also can be supplied as a list of vectors containing the indexes defining each fold as produced by split. If validation = "loo", leave-one-out cross-validation is performed. For fitted PLS-DA and sPLS-DA models, valid estimates the classification error rate using cross-validation. How many folds to generate is selected such that there is at least 1 sample for each class in the test set.

References

Tenenhaus, M. (1998). La r�gression PLS: th�orie et pratique. Paris: Editions Technic. L� Cao, K. A., Rossouw D., Robert-Grani�, C. and Besse, P. (2008). A sparse PLS for variable selection when integrating Omics data. Statistical Applications in Genetics and Molecular Biology 7, article 35. Mevik, B.-H., Cederkvist, H. R. (2004). Mean Squared Error of Prediction (MSEP) Estimates for Principal Component Regression (PCR) and Partial Least Squares Regression (PLSR). Journal of Chemometrics 18(9), 422-429.

Examples

Run this code

## validation for objects of class 'pls' (regression)
# ----------------------------------------
data(liver.toxicity)
X <- liver.toxicity$gene
Y <- liver.toxicity$clinic


# try tune the number of component to choose
# ---------------------
# first learn the full model
liver.pls <- pls(X, Y, ncomp = 10)

# with 5-fold cross validation: we use the same parameters as in model above
# but we perform cross validation to compute the MSEP, Q2 and R2 criteria
# ---------------------------
liver.val <- valid(liver.pls, validation = "Mfold", folds = 5)

# Q2 total should decrease until it reaches a threshold
liver.val$Q2.total

# ncomp = 3 is enough
plot(liver.val$Q2.total, type = 'l', col = 'red', ylim = c(-0.1, 0.5), 
	xlab = 'PLS components', ylab = 'Q2 total')
abline(h = 0.0975, col = 'darkgreen')
legend('topright', col = c('red', 'darkgreen'), legend = c('Q2 total', 'threshold 0.0975')
	, lty = 1)
title('Liver toxicity PLS 5-fold, Q2 values')

#have a look at the other criteria
# ----------------------
# R2
liver.val$R2
matplot(t(liver.val$R2), type = 'l', xlab = 'PLS components', ylab = 'R2 for each variable')
title('Liver toxicity PLS 5-fold, R2 values')

# MSEP
liver.val$MSEP
matplot(t(liver.val$MSEP), type = 'l', xlab = 'PLS components', ylab = 'MSEP for each variable')
title('Liver toxicity PLS 5-fold, MSEP values')


## validation for objects of class 'spls' (regression)
# ----------------------------------------
ncomp = 7
# first, learn the model on the whole data set
model.spls = spls(X, Y, ncomp = ncomp, mode = 'regression',
	 keepX = c(rep(5, ncomp)), keepY = c(rep(2,ncomp)))


# with leave-one-out cross validation
set.seed(45)
model.spls.loo.val <- valid(model.spls, validation = "loo")

#Q2 total
model.spls.loo.val$Q2.total

# R2:we can see how the performance degrades when ncomp increases
# results are similar to 5-fold
model.spls.loo.val$R2


## validation for objects of class 'splsda' (classification)
# ----------------------------------------
data(srbct)
X <- srbct$gene
Y <- srbct$class  

ncomp = 5

srbct.splsda <- splsda(X, Y, ncomp = ncomp, keepX = rep(10, ncomp))  

# with Mfold
# ---------
set.seed(45)
error <- valid(srbct.splsda, validation = "Mfold", folds = 8, 
               method = "all")

plot(error, type = "l")

Run the code above in your browser using DataLab