High-dimensional covariance estimation by minimizing \(\ell_1\)-penalized log-determinant divergence

High-dimensional covariance estimation by minimizing \(\ell_1\)-penalized log-determinant divergence

October 23, 2018 | Pradeep Ravikumar†, Martin J. Wainwright†, Garvesh Raskutti†, Bin Yu†,‡
This paper addresses the problem of estimating the covariance matrix and inverse covariance matrix (concentration matrix) of a high-dimensional random vector. The authors propose an estimator based on minimizing an $\ell_1$-penalized log-determinant Bregman divergence, which is equivalent to $\ell_1$-penalized maximum likelihood in the multivariate Gaussian case. They analyze the performance of this estimator under high-dimensional scaling, where the number of nodes $p$, the number of edges $s$, and the maximum node degree $d$ are allowed to grow with the sample size $n$. The analysis identifies key parameters that control the estimation rates, including the $\ell_\infty$-operator norm of the true covariance matrix, the $\ell_\infty$-operator norm of the sub-matrix $\Gamma_{SS}$, a mutual incoherence or irrepresentability measure on the matrix $\Gamma^*$, and the rate of decay of the tail probabilities of the sample covariance matrix. The main results establish consistency of the estimator in the elementwise $\ell_\infty$ norm and convergence rates in Frobenius and spectral norms. The authors also show that the estimator correctly specifies the zero pattern of the concentration matrix with high probability. The theoretical findings are supported by simulations for various graphs and problem parameters.This paper addresses the problem of estimating the covariance matrix and inverse covariance matrix (concentration matrix) of a high-dimensional random vector. The authors propose an estimator based on minimizing an $\ell_1$-penalized log-determinant Bregman divergence, which is equivalent to $\ell_1$-penalized maximum likelihood in the multivariate Gaussian case. They analyze the performance of this estimator under high-dimensional scaling, where the number of nodes $p$, the number of edges $s$, and the maximum node degree $d$ are allowed to grow with the sample size $n$. The analysis identifies key parameters that control the estimation rates, including the $\ell_\infty$-operator norm of the true covariance matrix, the $\ell_\infty$-operator norm of the sub-matrix $\Gamma_{SS}$, a mutual incoherence or irrepresentability measure on the matrix $\Gamma^*$, and the rate of decay of the tail probabilities of the sample covariance matrix. The main results establish consistency of the estimator in the elementwise $\ell_\infty$ norm and convergence rates in Frobenius and spectral norms. The authors also show that the estimator correctly specifies the zero pattern of the concentration matrix with high probability. The theoretical findings are supported by simulations for various graphs and problem parameters.
Reach us at info@study.space
Understanding High-dimensional covariance estimation by minimizing %E2%84%931-penalized log-determinant divergence