Model selection and estimation in regression with grouped variables

Model selection and estimation in regression with grouped variables

Received November 2004. Revised August 2005 | Ming Yuan and Yi Lin
This paper addresses the problem of selecting grouped variables (factors) for accurate prediction in regression, a common issue in multifactor analysis-of-variance (ANOVA) problems. The authors propose extensions of the lasso, LARS algorithm, and non-negative garrote to handle factor selection, aiming to improve upon traditional stepwise backward elimination methods. They introduce the group lasso, group LARS, and group non-negative garrote, and study their properties, including computational efficiency and performance. The group lasso and group LARS are shown to have piecewise linear solution paths, while the group non-negative garrote's solution path is generally not piecewise linear. The paper also introduces an approximate \(C_p\)-criterion for selecting the final model and compares the methods through simulations and a real example. The results indicate that the proposed methods outperform traditional stepwise methods in terms of prediction accuracy and computational efficiency.This paper addresses the problem of selecting grouped variables (factors) for accurate prediction in regression, a common issue in multifactor analysis-of-variance (ANOVA) problems. The authors propose extensions of the lasso, LARS algorithm, and non-negative garrote to handle factor selection, aiming to improve upon traditional stepwise backward elimination methods. They introduce the group lasso, group LARS, and group non-negative garrote, and study their properties, including computational efficiency and performance. The group lasso and group LARS are shown to have piecewise linear solution paths, while the group non-negative garrote's solution path is generally not piecewise linear. The paper also introduces an approximate \(C_p\)-criterion for selecting the final model and compares the methods through simulations and a real example. The results indicate that the proposed methods outperform traditional stepwise methods in terms of prediction accuracy and computational efficiency.
Reach us at info@study.space
Understanding Model selection and estimation in regression with grouped variables