Transfer Learning Under Large-Scale Low-Rank Regression Models

In high-dimensional multiple response regression problems, the large dimensionality of the coefficient matrix poses a challenge to parameter estimation. To address this challenge, low-rank matrix estimation methods have been developed to facilitate parameter estimation in the high-dimensional regime, where the number of parameters increases with sample size. Despite these methodological advances, accurately predicting multiple responses with limited target data remains a difficult task. To gain statistical power, the use of diverse datasets from source domains has emerged as a promising approach. In this article, we focus on the problem of transfer learning in a high-dimensional multiple response regression framework, which aims to improve estimation accuracy by transferring knowledge from informative source datasets. To reduce potential performance degradation due to the transfer of knowledge from irrelevant sources, we propose a novel transfer learning procedure including the forward selection of informative source sets. In particular, our forward source selection method is new compared to existing transfer learning framework, offering deeper theoretical insights and substantial methodological innovations. Theoretical results show that the proposed estimator achieves a faster convergence rate than the single-task penalized estimator using only target data. In addition, we develop an alternative transfer learning based on non-convex penalization to ensure rank consistency. Through simulations and real data experiments, we provide empirical evidence for the effectiveness of the proposed method and for its superiority over other methods. Supplementary materials for this article are available online, including a standardized description of the materials available for reproducing the work.