SPIKE AND SLAB VARIABLE SELECTION: FREQUENTIST AND BAYESIAN STRATEGIES

SPIKE AND SLAB VARIABLE SELECTION: FREQUENTIST AND BAYESIAN STRATEGIES

2005 | HEMANT ISHWARAN and J. SUNIL RAO
This paper introduces a rescaled spike and slab model for variable selection in linear regression, connecting it to frequentist generalized ridge regression. The model uses continuous bimodal priors for hypervariance parameters, enabling selective shrinkage of coefficients toward zero while retaining large values for nonzero coefficients. The rescaling of responses by √n ensures the prior has a nonvanishing effect, leading to a type of sample size universality. The model's posterior mean is shown to be asymptotically equivalent to the ordinary least squares (OLS) estimate under certain conditions, and it achieves oracle risk performance for model selection when combined with an appropriate prior. The paper also introduces a Zcut selection strategy, which uses a threshold based on a normal distribution to identify zero coefficients. The method is shown to outperform traditional OLS methods in terms of risk performance and model uncertainty reduction. Theoretical analysis connects the rescaled spike and slab model to generalized ridge regression, demonstrating its effectiveness in high-dimensional settings. The paper also discusses the importance of prior specifications and the role of rescaling in model selection. The results show that the rescaled spike and slab model provides a powerful Bayesian approach to variable selection, with applications in gene expression analysis and other high-dimensional regression problems.This paper introduces a rescaled spike and slab model for variable selection in linear regression, connecting it to frequentist generalized ridge regression. The model uses continuous bimodal priors for hypervariance parameters, enabling selective shrinkage of coefficients toward zero while retaining large values for nonzero coefficients. The rescaling of responses by √n ensures the prior has a nonvanishing effect, leading to a type of sample size universality. The model's posterior mean is shown to be asymptotically equivalent to the ordinary least squares (OLS) estimate under certain conditions, and it achieves oracle risk performance for model selection when combined with an appropriate prior. The paper also introduces a Zcut selection strategy, which uses a threshold based on a normal distribution to identify zero coefficients. The method is shown to outperform traditional OLS methods in terms of risk performance and model uncertainty reduction. Theoretical analysis connects the rescaled spike and slab model to generalized ridge regression, demonstrating its effectiveness in high-dimensional settings. The paper also discusses the importance of prior specifications and the role of rescaling in model selection. The results show that the rescaled spike and slab model provides a powerful Bayesian approach to variable selection, with applications in gene expression analysis and other high-dimensional regression problems.
Reach us at info@futurestudyspace.com
[slides] Spike and slab variable selection%3A Frequentist and Bayesian strategies | StudySpace