THE SPARSITY AND BIAS OF THE LASSO SELECTION IN HIGH-DIMENSIONAL LINEAR REGRESSION

THE SPARSITY AND BIAS OF THE LASSO SELECTION IN HIGH-DIMENSIONAL LINEAR REGRESSION

2008, Vol. 36, No. 4, 1567–1594 | BY CUN-HUI ZHANG1 AND JIAN HUANG2
The paper analyzes the sparsity and bias of the LASSO in high-dimensional linear regression. It shows that under a sparse Riesz condition on the design matrix, the LASSO selects a model with the correct order of dimensionality, controls the bias of the selected model, and selects all coefficients greater than a certain threshold. The results demonstrate that the LASSO is rate-consistent in model selection, meaning it achieves the best possible convergence rates for both the sum of squared errors and the $\ell_{\alpha}$-loss of the regression coefficients. The paper also discusses the implications of these results for the LASSO estimator, showing that it converges to the true mean and coefficients at the same rate as in the case of orthonormal designs. The sparse Riesz condition limits the range of eigenvalues of the covariance matrices of subsets of the design variables, ensuring that the LASSO can effectively select the correct model even when the number of variables is much larger than the sample size. The paper provides sufficient conditions for the sparse Riesz condition for both deterministic and random design matrices, and discusses the implications of these results for high-dimensional regression problems. The results are supported by theoretical proofs and numerical examples, demonstrating the effectiveness of the LASSO in high-dimensional settings.The paper analyzes the sparsity and bias of the LASSO in high-dimensional linear regression. It shows that under a sparse Riesz condition on the design matrix, the LASSO selects a model with the correct order of dimensionality, controls the bias of the selected model, and selects all coefficients greater than a certain threshold. The results demonstrate that the LASSO is rate-consistent in model selection, meaning it achieves the best possible convergence rates for both the sum of squared errors and the $\ell_{\alpha}$-loss of the regression coefficients. The paper also discusses the implications of these results for the LASSO estimator, showing that it converges to the true mean and coefficients at the same rate as in the case of orthonormal designs. The sparse Riesz condition limits the range of eigenvalues of the covariance matrices of subsets of the design variables, ensuring that the LASSO can effectively select the correct model even when the number of variables is much larger than the sample size. The paper provides sufficient conditions for the sparse Riesz condition for both deterministic and random design matrices, and discusses the implications of these results for high-dimensional regression problems. The results are supported by theoretical proofs and numerical examples, demonstrating the effectiveness of the LASSO in high-dimensional settings.
Reach us at info@study.space
Understanding The sparsity and bias of the Lasso selection in high-dimensional linear regression