VDA (version 1.3)

vda.le: Multicategory Vertex Discriminant Analysis (VDA) For High-Dimensional Data

Description

The method of vertex discriminant analysis (VDA) is ideally suited to handle multiple categories and an excess of predictors over training cases. vda.le is an elaboration of VDA that simultaneously conducts classification of $k$ possible categories and variable selection of p features, based on a data set of n cases. Variable selection is imposed using L1 (Lasso) and group Euclidean penalties. To use only Lasso penalization, set $\lambda2$=0. To use only Euclidean penalization, set $\lambda1$=0.

Usage

vda.le(x, y, lambda1, lambda2)

Arguments

x

n x p matrix or data frame containing the cases for each feature. The rows correspond to cases and the columns to the features. Intercept column is not included in this.

y

n x 1 vector representing the outcome variable. Each element denotes which one of the k classes that case belongs to

lambda1

Tuning parameter to control the lasso penalty. The default value is $1/n$. For determining the optimal lambda1, refer to cv.vda.le.

lambda2

Tuning parameter to control the Euclidean penalty. The default value is 0.01. For determining the optimal lambda1, refer to cv.vda.le

Value

feature

Feature matrix x with an intercept vector added as the first column. All entries in the first column should equal 1.

stand.feature

The feature matrix where the all columns are standardized, with the exception of the intercept column which is left unstandardized.

class

Class vector y. All elements should be integers between 1 and classes.

cases

Number of cases, n.

classes

Number of classes, k.

features

Number of feautres, p.

lambda

Vector of tuning constants where the first component is lambda1 and the second is lambda2

predicted

Vector of predicted category values based on VDA.

coefficient

The estimated coefficient matrix where the columns represent the coefficients for each predictor variable corresponding to k-1 outcome categories. The coefficient matrix is used for classifying new cases.

training_error_rate

The percentage of instances in the training set where the predicted outcome category is not equal to the case's true category.

nonzeros

Number of feature coefficients retained in the model. Is equal to p - number of features eliminated by penalization.

selected

An integer vector which represents the attributes that were selected after penalization.

call

The matched call.

Details

vda.le carries out cyclic coordinate descent in the context of VDA to minimize the loss function. By adding lasso ($L1$-norm) and group Euclidean penalties to the VDA loss function, unnecessary predictors are elimnated, adding parsimony and making the model more interpretable. Lasso penalties are applied to each predictor coefficient separately, while Euclidean penalties couples the coefficients of a single predictor and penalize the group collectively. If lambda1=0, then the overall penalty reduces to only group penalties. When lambda2=0, then the overall penalty reduces to the lasso. With these penalties in place, cyclic coordinate descent accelerates estimation of all coefficients.

References

Wu, T.T. and Lange, K. (2010) Multicategory Vertex Discriminant Analysis for High-Dimensional Data. Annals of Applied Statistics, Volume 4, No 4, 1698-1721. Lange, K. and Wu, T.T. (2008) An MM Algorithm for Multicategory Vertex Discriminant Analysis. Journal of Computational and Graphical Statistics, Volume 17, No 3, 527-544.

See Also

For determining the optimal values for lambda1 and lambda2, see cv.vda.le

For VDA without variable selection, see vda.r.

Examples

# load zoo data
# column 1 is name, columns 2:17 are features, column 18 is class
data(zoo)

#matrix containing all predictor vectors
x <- zoo[,2:17]

#outcome class vector
y <- zoo[,18]

#run VDA, Only Lasso Penalization, Set lambda2=0 
outlasso <-vda.le(x,y,lambda1=.02,lambda2=0)

#run VDA, Only Euclidean Penalization, Set lambda1=0 
outeuclid <-vda.le(x,y,lambda1=0,lambda2=0.04)

#run VDA, Lasso and Euclidean Penalization
outLE<-vda.le(x,y,lambda1=0.009,lambda2=0.05)
summary(outLE)

#Predict five cases based on VDA, Lasso and Euclidean Penalization
fivecases <- matrix(0,5,16)
fivecases[1,] <- c(1,0,0,1,0,0,0,1,1,1,0,0,4,0,1,0)
fivecases[2,] <- c(1,0,0,1,0,0,1,1,1,1,0,0,4,1,0,1)
fivecases[3,] <- c(0,1,1,0,1,0,0,0,1,1,0,0,2,1,1,0)
fivecases[4,] <- c(0,0,1,0,0,1,1,1,1,0,0,1,0,1,0,0)
fivecases[5,] <- c(0,0,1,0,0,0,1,0,0,0,0,0,0,0,0,0)
predict(outLE, fivecases)