sparseGAM (version 1.0)

cv.SSGL: Cross-Validation for Spike-and-Slab Group Lasso Regression

Description

This function implements \(K\)-fold cross-validation for group-regularized regression in the exponential dispersion family with the spike-and-slab group lasso (SSGL) penalty. The identity link function is used for Gaussian regression, the logit link is used for binomial regression, and the log link is used for Poisson, negative binomial, and gamma regression.

Usage

cv.SSGL(y, X, groups, 
        family=c("gaussian","binomial","poisson","negativebinomial","gamma"), 
        nb.size=1, gamma.shape=1, weights, nfolds=5, nlambda0=20,
        lambda0, lambda1, a, b, max.iter=100, tol=1e-6, print.fold=TRUE)

Arguments

y

\(n \times 1\) vector of responses.

X

\(n \times p\) design matrix, where the \(j\)th column of X corresponds to the \(j\)th overall covariate.

groups

\(p\)-dimensional vector of group labels. The \(j\)th entry in groups should contain either the group number or the name of the factor level that the \(j\)th covariate belongs to. groups must be either a vector of integers or factors.

family

exponential dispersion family. Allows for "gaussian", "binomial", "poisson", "negativebinomial", and "gamma". Note that for "negativebinomial", the size parameter must be specified, while for "gamma", the shape parameter must be specified.

nb.size

known size parameter \(\alpha\) in \(NB(\alpha,\mu_i)\) distribution for negative binomial responses. Default is nb.size=1. Ignored if family is not "negativebinomial".

gamma.shape

known shape parameter \(\nu\) in \(Gamma(\mu_i,\nu)\) distribution for gamma responses. Default is gamma.shape=1. Ignored if family is not "gamma".

weights

group-specific, nonnegative weights for the penalty. Default is to use the square roots of the group sizes.

nfolds

number of folds \(K\) to use in \(K\)-fold cross-validation. Default is nfolds=5.

nlambda0

number of spike hyperparameters \(L\). Default is nlambda0=20.

lambda0

grid of \(L\) spike hyperparameters \(\lambda_0\). The user may specify either a scalar or a vector. If the user does not provide this, the program chooses the grid automatically.

lambda1

slab hyperparameter \(\lambda_1\) in the SSGL prior. Default is lambda1=1.

a

shape hyperparameter for the \(Beta(a,b)\) prior on the mixing proportion in the SSGL prior. Default is a=1.

b

shape hyperparameter for the \(Beta(a,b)\) prior on the mixing proportion in the SSGL prior. Default is b=dim(X)[2].

max.iter

maximum number of iterations in the algorithm. Default is max.iter=100.

tol

convergence threshold for algorithm. Default is tol=1e-6.

print.fold

Boolean variable for whether or not to print the current fold in the algorithm. Default is print.fold=TRUE.

Value

The function returns a list containing the following components:

lambda0

\(L \times 1\) vector of spike hyperparameters lambda0 used to fit the model. lambda0 is displayed in descending order.

cve

\(L \times 1\) vector of mean cross-validation error across all \(K\) folds. The \(k\)th entry in cve corresponds to the \(k\)th regularization parameter in lambda0.

cvse

\(L \times 1\) vector of standard errors for cross-validation error across all \(K\) folds. The \(k\)th entry in cvse corresponds to the \(k\)th regularization parameter in lambda0.

lambda0.min

value of lambda0 that minimizes mean cross-validation error cve.

References

Bai R. (2021). "Spike-and-slab group lasso for consistent Bayesian estimation and variable selection in non-Gaussian generalized additive models." arXiv pre-print arXiv:2007.07021.

Bai, R., Moran, G. E., Antonelli, J. L., Chen, Y., and Boland, M.R. (2021). "Spike-and-slab group lassos for grouped regression and sparse generalized additive models." Journal of the American Statistical Association, in press.

Examples

# NOT RUN {
## Generate data
set.seed(12345)
X = matrix(runif(30*6), nrow=30)
n = dim(X)[1]
groups = c(1,1,1,2,2,3)
true.beta = c(-1.5,0.5,-1.5,0,0,0)

## Generate responses from Gaussian distribution
y = crossprod(t(X), true.beta) + rnorm(n)

## K-fold cross-validation for 3 choices of lambda0
## Note that if user does not specify lambda0, cv.SSGL chooses a grid automatically.

ssgl.mods = cv.SSGL(y, X, groups, family="gaussian", lambda0=seq(from=10,to=2,by=-4))

## Plot cross-validation curve
plot(ssgl.mods$lambda0, ssgl.mods$cve, type="l", xlab="lambda0", ylab="CVE")
## lambda which minimizes mean CVE
ssgl.mods$lambda0.min

# }
# NOT RUN {
## Example with Poisson regression

## Generate count responses
eta = crossprod(t(X), true.beta)
y = rpois(n,exp(eta))

## K-fold cross-validation with 4 choices of lambda0
## Note that if user does not specify lambda0, cv.SSGL chooses a grid automatically.

ssgl.poisson.mods = cv.SSGL(y, X, groups, family="poisson", lambda0=seq(from=8,to=2,by=-2))

## Plot cross-validation curve
plot(ssgl.poisson.mods$lambda0, ssgl.poisson.mods$cve, type="l", xlab="lambda0", ylab="CVE")
## lambda which minimizes mean CVE
ssgl.poisson.mods$lambda0.min
# }