Shrinking the Covariance Matrix Using Convex Penalties on the Matrix-Log Transformation

For <i>q</i>-dimensional data, penalized versions of the sample covariance matrix are important when the sample size is small or modest relative to <i>q</i>. Since the negative log-likelihood under multivariate normal sampling is convex in Σ−1, the inverse of the covariance matrix, it is common to consider additive penalties which are also convex in Σ−1. More recently, Deng and Tsui and Yu et al. have proposed penalties which are strictly functions of the roots of Σ and are convex in log Σ, but not in Σ−1. The resulting penalized optimization problems, though, are neither convex in log Σ nor in Σ−1. In this article, however, we show these penalized optimization problems to be geodesically convex in Σ. This allows us to establish the existence and uniqueness of the corresponding penalized covariance matrices. More generally, we show that geodesic convexity in Σ is equivalent to convexity in log Σ for penalties which are functions of the roots of Σ. In addition, when using such penalties, the resulting penalized optimization problem reduces to a <i>q</i>-dimensional convex optimization problem on the logs of the roots of Σ, which can then be readily solved via Newton’s algorithm. Supplementary materials for this article are available online.

针对q维数据，当样本量相较于q偏小或适中时，带惩罚项的样本协方差矩阵（sample covariance matrix）具有重要的理论与应用价值。由于在多元正态抽样（multivariate normal sampling）下，负对数似然（negative log-likelihood）关于协方差矩阵的逆Σ⁻¹是凸函数，因此学界通常会考虑同样关于Σ⁻¹为凸的加性惩罚项（additive penalties）。近年来，Deng与Tsui以及Yu等人提出了一类新型惩罚项，这类惩罚项仅为Σ的特征根（roots of Σ）的函数，且关于对数协方差矩阵log Σ为凸，但关于Σ⁻¹并不凸。不过由此得到的带惩罚项优化问题，既不关于log Σ凸，也不关于Σ⁻¹凸。然而在本文中，我们证明了这类带惩罚项优化问题关于Σ具有测地凸性（geodesically convex）。这一结论使得我们能够严格证明对应带惩罚协方差矩阵的存在性与唯一性。更一般地，我们证明了：对于仅为Σ的特征根的函数的惩罚项而言，关于Σ的测地凸性与关于log Σ的凸性完全等价。此外，当采用这类惩罚项时，原带惩罚项优化问题可简化为关于Σ的特征根对数的q维凸优化问题，进而可通过牛顿算法（Newton’s algorithm）快速求解。本文的补充材料可在线获取。