Uncertainty Quantification for Computer Models With Spatial Output Using Calibration-Optimal Bases

The calibration of complex computer codes using uncertainty quantification (UQ) methods is a rich area of statistical methodological development. When applying these techniques to simulators with spatial output, it is now standard to use principal component decomposition to reduce the dimensions of the outputs in order to allow Gaussian process emulators to predict the output for calibration. We introduce the “terminal case,” in which the model cannot reproduce observations to within model discrepancy, and for which standard calibration methods in UQ fail to give sensible results. We show that even when there is no such issue with the model, the standard decomposition on the outputs can and usually does lead to a terminal case analysis. We present a simple test to allow a practitioner to establish whether their experiment will result in a terminal case analysis, and a methodology for defining calibration-optimal bases that avoid this whenever it is not inevitable. We present the optimal rotation algorithm for doing this, and demonstrate its efficacy for an idealized example for which the usual principal component methods fail. We apply these ideas to the CanAM4 model to demonstrate the terminal case issue arising for climate models. We discuss climate model tuning and the estimation of model discrepancy within this context, and show how the optimal rotation algorithm can be used in developing practical climate model tuning tools. Supplementary materials for this article are available online.

利用不确定性量化（uncertainty quantification, UQ）方法开展复杂计算机代码的校准研究，是统计方法论发展的一个活跃领域。当将此类技术应用于带有空间输出的模拟器时，当前的标准流程是采用主成分分解（principal component decomposition）对输出维度进行降维，以便高斯过程模拟器（Gaussian process emulators）能够为校准任务预测输出结果。我们提出了“终端情形（terminal case）”这一概念：当模型无法在模型偏差范围内复现观测值时，常规的UQ校准方法无法得到合理的结果。我们证明，即便模型不存在上述问题，对输出进行的标准主成分分解也可能且通常会触发终端情形分析。我们提出了一项简易测试，可帮助研究人员判断其开展的实验是否会出现终端情形；同时给出了一套用于定义校准最优基的方法论，在非不可避免的情况下可规避该问题。我们提出了用于实现该目标的最优旋转算法（optimal rotation algorithm），并通过一个理想示例验证了其有效性——该示例中常规主成分方法会失效。我们将这些思路应用于CanAM4模型，以展示气候模型中存在的终端情形问题。我们讨论了气候模型调优以及该背景下的模型偏差估计，并阐明了最优旋转算法如何可用于开发实用的气候模型调优工具。本文的补充材料可在线获取。