This paper addresses the estimation of a high-dimensional covariance matrix with a conditional sparsity structure and fast-diverging eigenvalues. The authors introduce the Principal Orthogonal Complement Thresholding (POET) method, which explores the approximate factor structure with sparsity. POET includes several specific estimators such as the sample covariance matrix, the factor-based covariance matrix, the thresholding estimator, and the adaptive thresholding estimator. The paper provides mathematical insights into the relationship between factor analysis and principal component analysis in high-dimensional settings. It studies the convergence rates of the sparse residual covariance matrix and the conditional sparse covariance matrix under various norms. The impact of estimating unknown factors on the convergence rate is shown to vanish as the dimensionality increases. The uniform rates of convergence for unobserved factors and their factor loadings are derived, and these results are verified through extensive simulation studies. A real data application on portfolio allocation is also presented.This paper addresses the estimation of a high-dimensional covariance matrix with a conditional sparsity structure and fast-diverging eigenvalues. The authors introduce the Principal Orthogonal Complement Thresholding (POET) method, which explores the approximate factor structure with sparsity. POET includes several specific estimators such as the sample covariance matrix, the factor-based covariance matrix, the thresholding estimator, and the adaptive thresholding estimator. The paper provides mathematical insights into the relationship between factor analysis and principal component analysis in high-dimensional settings. It studies the convergence rates of the sparse residual covariance matrix and the conditional sparse covariance matrix under various norms. The impact of estimating unknown factors on the convergence rate is shown to vanish as the dimensionality increases. The uniform rates of convergence for unobserved factors and their factor loadings are derived, and these results are verified through extensive simulation studies. A real data application on portfolio allocation is also presented.