HIGH-DIMENSIONAL GRAPHS AND VARIABLE SELECTION WITH THE LASSO

HIGH-DIMENSIONAL GRAPHS AND VARIABLE SELECTION WITH THE LASSO

2006, Vol. 34, No. 3, 1436-1462 | BY NICOLAI MEINSHAUSEN AND PETER BÜHLMANN
The article introduces a computationally efficient method for variable selection in high-dimensional graphs using the Lasso. It demonstrates that neighborhood selection with the Lasso is a viable alternative to traditional covariance selection for sparse graphs, where the number of variables can grow as a power of the number of observations. The method estimates conditional independence structures by analyzing the neighborhood of each node, equivalent to variable selection in Gaussian linear models. The Lasso approach is consistent for sparse graphs under specific assumptions, such as sparsity and bounded partial correlations. The key insight is that the penalty parameter must be chosen carefully to avoid falsely connecting distinct connectivity components of the graph, ensuring exponential convergence to the true neighborhood. Unlike the "oracle" penalty, which is inconsistent for model selection, the proposed penalty controls false positives, enabling reliable estimation even when the number of variables is much larger than the number of observations. The method is computationally efficient, handling graphs with thousands of nodes using only a few hundred observations, outperforming traditional forward selection MLE in accuracy and speed.The article introduces a computationally efficient method for variable selection in high-dimensional graphs using the Lasso. It demonstrates that neighborhood selection with the Lasso is a viable alternative to traditional covariance selection for sparse graphs, where the number of variables can grow as a power of the number of observations. The method estimates conditional independence structures by analyzing the neighborhood of each node, equivalent to variable selection in Gaussian linear models. The Lasso approach is consistent for sparse graphs under specific assumptions, such as sparsity and bounded partial correlations. The key insight is that the penalty parameter must be chosen carefully to avoid falsely connecting distinct connectivity components of the graph, ensuring exponential convergence to the true neighborhood. Unlike the "oracle" penalty, which is inconsistent for model selection, the proposed penalty controls false positives, enabling reliable estimation even when the number of variables is much larger than the number of observations. The method is computationally efficient, handling graphs with thousands of nodes using only a few hundred observations, outperforming traditional forward selection MLE in accuracy and speed.
Reach us at info@study.space