A Scalable Method to Exploit Screening in Gaussian Process Models with Noise

A common approach to approximating Gaussian log-likelihoods at scale exploits the fact that precision matrices can be well-approximated by sparse matrices in some circumstances. This strategy is motivated by the <i>screening effect</i>, which refers to the phenomenon in which the linear prediction of a process <i>Z</i> at a point x0 depends primarily on measurements nearest to x0. But simple perturbations, such as iid measurement noise, can significantly reduce the degree to which this exploitable phenomenon occurs. While strategies to cope with this issue already exist and are certainly improvements over ignoring the problem, in this work we present a new one based on the EM algorithm that offers several advantages. While in this work we focus on the application to Vecchia’s approximation (Vecchia), a particularly popular and powerful framework in which we can demonstrate true second-order optimization of M steps, the method can also be applied using entirely matrix-vector products, making it applicable to a very wide class of precision matrix-based approximation methods. Supplementary materials for this article are available online.

面向大规模高斯对数似然近似的常用方法，利用了特定条件下精度矩阵（precision matrices）可通过稀疏矩阵（sparse matrices）实现良好近似这一性质。该策略的提出源于**筛选效应（screening effect）**：其指代的是，对某点x₀处的过程Z进行线性预测时，预测结果主要依赖于距x₀最近的观测值这一现象。但诸如独立同分布（iid）测量噪声这类简单扰动，会显著削弱这一可被利用的效应的表现程度。尽管目前已有针对该问题的应对策略，且相较于忽视该问题而言已有明显改进，但本文提出了一种基于期望最大化（Expectation-Maximization, EM）算法的新方法，该方法具备多项优势。尽管本文的研究重点聚焦于Vecchia近似（Vecchia）这一在领域内广受欢迎且功能强大的框架——在此框架中我们可实现M步的真正二阶优化——但该方法同样可完全通过矩阵-向量积（matrix-vector products）实现应用，因此适用于绝大多数基于精度矩阵的近似方法。本文补充材料可在线获取。