MCMCSVDreg: Markov Chain Monte Carlo for SVD Regression

Description

This function generates a sample from the posterior distribution of a linear regression model with Gaussian errors in which the design matrix has been decomposed with singular value decomposition.The sampling is done via the Gibbs sampling algorithm. The user supplies data and priors, and a sample from the posterior distribution is returned as an mcmc object, which can be subsequently analyzed with functions provided in the coda package.

Usage

MCMCSVDreg(
  formula,
  data = NULL,
  burnin = 1000,
  mcmc = 10000,
  thin = 1,
  verbose = 0,
  seed = NA,
  tau2.start = 1,
  g0 = 0,
  a0 = 0.001,
  b0 = 0.001,
  c0 = 2,
  d0 = 2,
  w0 = 1,
  beta.samp = FALSE,
  intercept = TRUE,
  ...
)

Value

An mcmc object that contains the posterior sample. This object can be summarized by functions provided by the coda package.

Arguments

formula: Model formula. Predictions are returned for elements of y that are coded as NA.
data: Data frame.
burnin: The number of burn-in iterations for the sampler.
mcmc: The number of MCMC iterations after burnin.
thin: The thinning interval used in the simulation. The number of MCMC iterations must be divisible by this value.
verbose: A switch which determines whether or not the progress of the sampler is printed to the screen. If verbose is greater than 0 the iteration number, the $\beta$ vector, and the error variance are printed to the screen every verboseth iteration.
seed: The seed for the random number generator. If NA, the Mersenne Twister generator is used with default seed 12345; if an integer is passed it is used to seed the Mersenne twister. The user can also pass a list of length two to use the L'Ecuyer random number generator, which is suitable for parallel computation. The first element of the list is the L'Ecuyer seed, which is a vector of length six or NA (if NA a default seed of rep(12345,6) is used). The second element of list is a positive substream number. See the MCMCpack specification for more details.
tau2.start: The starting values for the $\tau^2$ vector. Can be either a scalar or a vector. If a scalar is passed then that value will be the starting value for all elements of $\tau^2$.
g0: The prior mean of $\gamma$. This can either be a scalar or a column vector with dimension equal to the number of gammas. If this takes a scalar value, then that value will serve as the prior mean for all of the betas.
a0: $a_0/2$ is the shape parameter for the inverse Gamma prior on $\sigma^2$ (the variance of the disturbances). The amount of information in the inverse Gamma prior is something like that from $a_0$ pseudo-observations.
b0: $b_0/2$ is the scale parameter for the inverse Gamma prior on $\sigma^2$ (the variance of the disturbances). In constructing the inverse Gamma prior, $b_0$ acts like the sum of squared errors from the $a_0$ pseudo-observations.
c0: $c_0/2$ is the shape parameter for the inverse Gamma prior on $\tau_i^2$.
d0: $d_0/2$ is the scale parameter for the inverse Gamma prior on $\tau_i^2$.
w0: The prior probability that $\gamma_i = 0$. Can be either a scalar or an $N$ vector where $N$ is the number of observations.
beta.samp: Logical indicating whether the sampled elements of beta should be stored and returned.
intercept: Logical indicating whether the original design matrix should include a constant term.
...: further arguments to be passed

Details

The model takes the following form: $$y = X \beta + \varepsilon$$ Where the errors are assumed to be iid Gaussian: $$\varepsilon_{i} \sim \mathcal{N}(0, \sigma^2)$$

Let $N$ denote the number of rows of $X$ and $P$ the number of columns of $X$. Unlike the standard regression setup where $N >> P$ here it is the case that $P >> N$.

To deal with this problem a singular value decomposition of $X'$ is performed: $X' = ADF$ and the regression model becomes

$$y = F'D \gamma + \varepsilon$$

where $\gamma = A' \beta$

We assume the following priors:

$$\sigma^{-2} \sim \mathcal{G}amma(a_0/2, b_0/2)$$

$$\tau^{-2} \sim \mathcal{G}amma(c_0/2, d_0/2)$$

$$\gamma_i \sim w0_i \delta_0 + (1-w0_i) \mathcal{N}(g0_i, \sigma^2 \tau_i^2/ d_i^2)$$

where $\delta_0$ is a unit point mass at 0 and $d_i$ is the $i$th diagonal element of $D$.

References

Mike West, Josheph Nevins, Jeffrey Marks, Rainer Spang, and Harry Zuzan. 2000. ``DNA Microarray Data Analysis and Regression Modeling for Genetic Expression Profiling." Duke ISDS working paper.

Gottardo, Raphael, and Adrian Raftery. 2004. ``Markov chain Monte Carlo with mixtures of singular distributions.'' Statistics Department, University of Washington, Technical Report 470.

Andrew D. Martin, Kevin M. Quinn, and Jong Hee Park. 2011. ``MCMCpack: Markov Chain Monte Carlo in R.'', Journal of Statistical Software. 42(9): 1-21. tools:::Rd_expr_doi("10.18637/jss.v042.i09").

Daniel Pemstein, Kevin M. Quinn, and Andrew D. Martin. 2007. Scythe Statistical Library 1.0. http://scythe.wustl.edu.s3-website-us-east-1.amazonaws.com/.

Martyn Plummer, Nicky Best, Kate Cowles, and Karen Vines. 2006. ``Output Analysis and Diagnostics for MCMC (CODA)'', R News. 6(1): 7-11. https://CRAN.R-project.org/doc/Rnews/Rnews_2006-1.pdf.