Sample efficient nonparametric regression via low-rank regularization

Nonparametric regression suffers from curse of dimensionality, requiring a relatively large sample size for accurate estimation beyond the univariate case. In this paper, we consider a simple method of dimension reduction in nonparametric regression via series estimation, based on the concept of low-rankness which was previously studied in parametric multivariate reduced-rank regression and matrix regression. For d>2, the low-rank assumption is realized via tensor regression. We establish a faster convergence rate of the estimator in the (approximate) low-rank case. Limitations of the model are also discussed. Through simulation studies and real data analysis, we compare the estimation accuracy of the proposed method with that of existing approaches. The results demonstrate that the proposed method yields estimates with lower RMSE compared to existing methods.

非参数回归存在维数灾难（curse of dimensionality）问题，在单变量情形之外开展准确估计时，需要相对较大的样本量。本文基于此前在参数化多变量降秩回归与矩阵回归中研究过的低秩性（low-rankness）概念，提出一种通过级数估计实现非参数回归降维的简便方法。当维度d>2时，该低秩假设通过张量回归（tensor regression）得以实现。本文证明了，在（近似）低秩情形下，所提估计量具备更快的收敛速度。本文还讨论了该模型的局限性。通过模拟实验与真实数据分析，本文将所提方法的估计精度与现有方法进行了对比。结果表明，相较于现有方法，本文所提方法得到的估计结果具有更低的均方根误差（Root Mean Square Error，RMSE）。