Optimal Plug-in Gaussian Processes for Modeling Derivatives

Derivatives are a key nonparametric functional in wide-ranging applications where the rate of change of an unknown function is of interest. In the Bayesian paradigm, Gaussian processes (GPs) are routinely used as a flexible prior for unknown functions, and are arguably one of the most popular tools in many areas. However, little is known about the optimal modeling strategy and theoretical properties when using GPs for derivatives. In this article, we study a plug-in strategy by differentiating the posterior distribution with GP priors for derivatives of any order. This practically appealing plug-in GP method has been previously perceived as suboptimal and degraded, but this is not necessarily the case. We provide posterior contraction rates for plug-in GPs and establish that they achieve optimal rates simultaneously for all derivative orders. We show that the posterior measure of the regression function and its derivatives, with the same choice of hyperparameter that does not depend on the order of derivatives, converges at the minimax optimal rate up to a logarithmic factor for functions in certain classes. We analyze a data-driven hyperparameter tuning method based on empirical Bayes, and show that it satisfies the optimal rate condition while maintaining computational efficiency. This article to the best of our knowledge provides the first positive result for plug-in GPs in the context of inferring derivative functionals, and leads to a practically simple nonparametric Bayesian method with optimal and adaptive hyperparameter tuning for simultaneously estimating the regression function and its derivatives. Simulations show competitive finite sample performance of the plug-in GP method. A climate change application for analyzing the global sea-level rise is discussed. Supplementary materials for this article are available online, including a standardized description of the materials available for reproducing the work.导数是一类关键的非参数泛函，广泛应用于关注未知函数变化率的各类场景中。在贝叶斯框架下，高斯过程（Gaussian Processes）常被用作未知函数的灵活先验分布，堪称众多领域中最受欢迎的建模工具之一。然而，针对利用高斯过程进行导数建模的最优策略与理论性质，目前学界尚缺乏系统研究。本文针对任意阶导数，通过对配备高斯过程先验的后验分布进行求导，研究了一种插补式建模策略。这种在实践中颇具吸引力的插补式高斯过程方法，此前被认为存在次优性与性能退化问题，但实际情况未必如此。我们推导了插补式高斯过程的后验收缩率，并证明该方法可同时针对所有导数阶数达到最优收缩率。我们证明，对于特定函数类，当采用不依赖导数阶数的统一超参数选择时，回归函数及其导数的后验分布可在至多相差一个对数因子的范围内达到极小极大最优收敛率。我们分析了一种基于经验贝叶斯的数据驱动超参数调优方法，并证明该方法在保证计算效率的同时，可满足最优率条件。据我们所知，本文首次在导数泛函推断场景下为插补式高斯过程方法提供了正向理论结果，并提出了一种实践中简便易行的非参数贝叶斯方法，该方法可通过最优且自适应的超参数调优，同时实现回归函数及其导数的估计。仿真实验表明，该插补式高斯过程方法的有限样本性能颇具竞争力。本文还讨论了一个用于分析全球海平面上升的气候变化应用案例。本文的补充材料可在线获取，其中包含可用于复现研究成果的标准化材料说明。