High-Dimensional Multivariate Linear Regression with Weighted Nuclear Norm Regularization
收藏资源简介：
We consider a low-rank matrix estimation problem when the data is assumed to be generated from the multivariate linear regression model. To induce the low-rank coefficient matrix, we employ the weighted nuclear norm (WNN) penalty defined as the weighted sum of the singular values of the matrix. The weights are set in a nondecreasing order, which yields the non-convexity of the WNN objective function in the parameter space. Although the objective function has been widely applied, studies on the estimation properties of its resulting estimator are limited. We propose an efficient algorithm under the framework of the alternative directional method of multipliers (ADMM) to estimate the coefficient matrix. The estimator from the suggested algorithm converges to a stationary point of an augmented Lagrangian function. Under the orthogonal design setting, the effects of the weights for estimating the singular values of the ground-truth coefficient matrix are derived. Under the Gaussian design setting, a minimax convergence rate on the estimation error is derived. We also propose a generalized cross-validation (GCV) criterion for selecting the tuning parameter and an iterative algorithm for updating the weights. Simulations and a real data analysis demonstrate the competitive performance of our new method. Supplementary materials for this article are available online.
本文研究数据服从多元线性回归模型生成时的低秩矩阵估计问题。为诱导得到低秩系数矩阵，我们采用加权核范数（weighted nuclear norm, WNN）惩罚项，其定义为矩阵奇异值的加权和。将惩罚权重设置为非递减序列时，加权核范数目标函数在参数空间中呈现非凸性。尽管该目标函数已得到广泛应用，但针对其导出估计量的统计性质的研究仍较为有限。我们基于交替方向乘子法（alternating direction method of multipliers, ADMM）框架提出了一种高效的系数矩阵估计算法，该算法得到的估计量可收敛至增广拉格朗日函数的驻点。在正交设计场景下，我们推导得到了权重对真实系数矩阵奇异值估计的影响规律；在高斯设计场景下，我们推导得到了估计误差的极小极大收敛速率。此外，我们还提出了用于选择调节参数的广义交叉验证（generalized cross-validation, GCV）准则，以及用于更新权重的迭代算法。仿真实验与真实数据分析结果表明，所提新方法具备颇具竞争力的性能。本文的补充材料可在线获取。
