This paper investigates the consistency and sparsity of principal components analysis (PCA) in high-dimensional settings. The authors argue that when the number of variables $ p $ is comparable to or larger than the number of observations $ n $, a preliminary reduction in dimensionality is necessary before applying PCA. They propose that this reduction can be achieved by working in a basis where the signals have a sparse representation.
The paper presents an asymptotic model where the estimate of the leading principal component vector via standard PCA is consistent if and only if $ p(n)/n \rightarrow 0 $. A simple algorithm is described for selecting a subset of coordinates with the largest sample variances, and it is shown that if PCA is performed on this subset, consistency is recovered even when $ p(n) \gg n $.
The authors also discuss the implications of the inconsistency of standard PCA when $ p $ is comparable to $ n $, and they show that the consistency of PCA can be recovered by working in a basis where the signals have a sparse representation. They provide theoretical results and an illustrative algorithm for sparse PCA, which involves selecting a subset of variables with the largest variances, performing PCA on this subset, and then thresholding the resulting eigenvectors to remove noise.
The paper also discusses the computational complexity of sparse PCA and provides examples demonstrating its effectiveness in reducing noise while preserving the main features of the data. The authors conclude that sparse PCA is a more effective method for high-dimensional data than standard PCA, as it reduces noise while preserving the main features of the data and uses significantly less computational time.This paper investigates the consistency and sparsity of principal components analysis (PCA) in high-dimensional settings. The authors argue that when the number of variables $ p $ is comparable to or larger than the number of observations $ n $, a preliminary reduction in dimensionality is necessary before applying PCA. They propose that this reduction can be achieved by working in a basis where the signals have a sparse representation.
The paper presents an asymptotic model where the estimate of the leading principal component vector via standard PCA is consistent if and only if $ p(n)/n \rightarrow 0 $. A simple algorithm is described for selecting a subset of coordinates with the largest sample variances, and it is shown that if PCA is performed on this subset, consistency is recovered even when $ p(n) \gg n $.
The authors also discuss the implications of the inconsistency of standard PCA when $ p $ is comparable to $ n $, and they show that the consistency of PCA can be recovered by working in a basis where the signals have a sparse representation. They provide theoretical results and an illustrative algorithm for sparse PCA, which involves selecting a subset of variables with the largest variances, performing PCA on this subset, and then thresholding the resulting eigenvectors to remove noise.
The paper also discusses the computational complexity of sparse PCA and provides examples demonstrating its effectiveness in reducing noise while preserving the main features of the data. The authors conclude that sparse PCA is a more effective method for high-dimensional data than standard PCA, as it reduces noise while preserving the main features of the data and uses significantly less computational time.