Spectrally Sparse Nonparametric Regression via Elastic Net Regularized Smoothers

Nonparametric regression frameworks, such as generalized additive models (GAMs) and smoothing spline analysis of variance (SSANOVA) models, extend the generalized linear model (GLM) by allowing for unknown functional relationships between an exponential family response variable and a collection of predictor variables. The unknown functional relationships are typically estimated using penalized likelihood estimation, which adds a roughness penalty to the (negative) log-likelihood function. In this article, I propose a spectral parameterization of a smoothing spline, which allows for an efficient application of Elastic Net regression to smooth and select eigenvectors of a kernel matrix. The classic (ridge regression) solution for a smoothing spline is a special case of the proposed kernel eigenvector smoothing and selection operator. Extensions for tensor product smoothers are developed for both the GAM and SSANOVA frameworks. Using simulated and real data examples, I demonstrate that the proposed approach offers practical and computational gains over typical approaches for fitting GAMs, SSANOVA models, and Elastic Net penalized GLMs. Supplementary materials for this article are available online.非参数回归框架（Nonparametric regression frameworks），如广义加性模型（generalized additive models, GAMs）与平滑样条方差分析（smoothing spline analysis of variance, SSANOVA）模型，通过允许指数族响应变量与一组预测变量间存在未知函数关系，对广义线性模型（generalized linear model, GLM）进行了拓展。这类未知的函数关系通常借助惩罚似然估计（penalized likelihood estimation）进行拟合，该方法会向（负）对数似然函数（log-likelihood function）添加粗糙度惩罚项。本文提出了一种平滑样条的谱参数化方法，可高效应用弹性网回归（Elastic Net regression）对核矩阵（kernel matrix）的特征向量进行平滑与选择。经典的岭回归（ridge regression）平滑样条解法，是本文提出的核特征向量平滑与选择算子的特例。针对广义加性模型与平滑样条方差分析框架，本文还拓展了张量积平滑器（tensor product smoothers）的相关应用。通过模拟数据与真实数据示例，本文证明所提方法在拟合广义加性模型、平滑样条方差分析模型以及弹性网惩罚广义线性模型时，兼具实用价值与计算优势。本文的补充材料可在线获取。