C-ISTA: Iterative Shrinkage-Thresholding Algorithm for Sparse Covariance Matrix Estimation

Covariance matrix estimation is a fundamental task in many fields related to data analysis. As the dimension of the covariance matrix becomes large, it is desirable to obtain a sparse estimator and an efficient algorithm to compute it. In this paper, we consider the covariance matrix estimation problem by minimizing a Gaussian negative log-likelihood loss function with an ℓ1 penalty, which is a constrained non-convex optimization problem. We propose to solve the covariance estimator via a simple iterative shrinkage-thresholding algorithm (C-ISTA) with provable convergence. Numerical simulations with comparison to the benchmark methods demonstrate the computational efficiency and good estimation performance of C-ISTA.

I. Introduction

Estimation of a covariance matrix is one of the fundamental problems in many research areas, e.g., biology [1], finance [2]–[4], signal processing [5], [6], machine learning [7], etc. The sample covariance matrix (SCM) could be one of the most commonly used estimators, which is computationally simple and consistent with the Gaussian maximum likelihood estimator (MLE). In covariance estimation, the number of parameters to be estimated is the square of the variable dimension. When the dimension is high, the SCM estimator can perform badly with a limited number of observations [8].