Spectrally Sparse Nonparametric Regression via Elastic Net Regularized Smoothers

Nonparametric regression frameworks, such as generalized additive models (GAMs) and smoothing spline analysis of variance (SSANOVA) models, extend the generalized linear model (GLM) by allowing for unknown functional relationships between an exponential family response variable and a collection of predictor variables. The unknown functional relationships are typically estimated using penalized likelihood estimation, which adds a roughness penalty to the (negative) log-likelihood function. In this article, I propose a spectral parameterization of a smoothing spline, which allows for an efficient application of Elastic Net regression to smooth and select eigenvectors of a kernel matrix. The classic (ridge regression) solution for a smoothing spline is a special case of the proposed kernel eigenvector smoothing and selection operator. Extensions for tensor product smoothers are developed for both the GAM and SSANOVA frameworks. Using simulated and real data examples, I demonstrate that the proposed approach offers practical and computational gains over typical approaches for fitting GAMs, SSANOVA models, and Elastic Net penalized GLMs. Supplementary materials for this article are available online.

非参数回归框架（如广义加性模型（generalized additive models, GAMs）和平滑样条方差分析（smoothing spline analysis of variance, SSANOVA）模型）通过允许指数族响应变量与一组预测变量之间存在未知函数关系，对广义线性模型（generalized linear model, GLM）进行了拓展。这类未知的函数关系通常通过惩罚似然估计进行求解，该方法会向负对数似然函数中加入粗糙度惩罚项。本文提出了一种平滑样条的谱参数化方法，可实现弹性网回归（Elastic Net regression）高效应用于核矩阵（kernel matrix）特征向量的光滑化与选择。平滑样条的经典岭回归（ridge regression）解，是所提出的核特征向量光滑与选择算子的特例。针对GAM与SSANOVA框架，本文分别拓展了张量积光滑器（tensor product smoothers）的相关方法。通过模拟数据与真实数据示例，本文证明所提方法相较于拟合GAMs、SSANOVA模型以及弹性网惩罚广义线性模型的常规方法，在实用性与计算效率层面均具备优势。本文的补充材料可在线获取。