Transfer learning under large-scale low-rank regression models

In high-dimensional multiple response regression problems, the large dimensionality of the coefficient matrix poses a challenge to parameter estimation. To address this challenge, low-rank matrix estimation methods have been developed to facilitate parameter estimation in the high-dimensional regime, where the number of parameters increases with sample size. Despite these methodological advances, accurately predicting multiple responses with limited target data remains a difficult task. To gain statistical power, the use of diverse datasets from source domains has emerged as a promising approach. In this paper, we focus on the problem of transfer learning in a high-dimensional multiple response regression framework, which aims to improve estimation accuracy by transferring knowledge from informative source datasets. To reduce potential performance degradation due to the transfer of knowledge from irrelevant sources, we propose a novel transfer learning procedure including the forward selection of informative source sets. In particular, our forward source selection method is new compared to existing transfer learning framework, offering deeper theoretical insights and substantial methodological innovations. Theoretical results show that the proposed estimator achieves a faster convergence rate than the single-task penalized estimator using only target data. In addition, we develop an alternative transfer learning based on non-convex penalization to ensure rank consistency. Through simulations and real data experiments, we provide empirical evidence for the effectiveness of the proposed method and for its superiority over other methods.

在高维多响应回归（high-dimensional multiple response regression）问题中，系数矩阵的高维特性给参数估计（parameter estimation）带来了挑战。为应对这一挑战，学界已发展出低秩矩阵估计（low-rank matrix estimation）方法，以在参数数量随样本量（sample size）增长的高维情形（high-dimensional regime）下辅助参数估计。尽管方法学上已有这些进展，但利用有限的目标数据精准预测多响应变量仍是一项艰巨任务。为提升统计功效（statistical power），从源域获取多样化数据集的思路已成为颇具前景的解决方案。
本文聚焦于高维多响应回归框架下的迁移学习（transfer learning）问题，旨在通过从信息丰富的源数据集（source datasets）迁移知识来提升估计精度。为缓解因从无关源迁移知识所引发的性能退化问题，我们提出了一种全新的迁移学习流程，其中包含信息源集的向前选择（forward selection of informative source sets）步骤。具体而言，相较于现有迁移学习框架，我们提出的向前源选择方法具备更深入的理论内涵与显著的方法学创新。
理论结果表明，相较于仅使用目标数据的单任务惩罚估计量（single-task penalized estimator），本文所提估计器可实现更快的收敛速率。此外，我们还开发了一种基于非凸惩罚（non-convex penalization）的替代迁移学习方法，以确保秩一致性（rank consistency）。通过模拟实验与真实数据试验，我们为所提方法的有效性及其相较于其他方法的优越性提供了实证依据。