ModelParam.R from The importance of uncertainty quantification in model reproducibility

Many computer models possess high-dimensional input spaces and substantial computational time to produce a single model evaluation. Although such models are often ‘deterministic’, these models suffer from a wide range of uncertainties. We argue that uncertainty quantification is crucial for computer model validation and reproducibility. We present a statistical framework, termed history matching, for performing global parameter search by comparing model output to the observed data. We employ Gaussian process (GP) emulators to produce fast predictions about model behaviour at the arbitrary input parameter settings allowing output uncertainty distributions to be calculated. History matching identifies sets of input parameters that give rise to acceptable matches between observed data and model output given our representation of uncertainties. Modellers could proceed by simulating computer models’ outputs of interest at these identified parameter settings and producing a range of predictions. The variability in model results is crucial for inter-model comparison as well as model development. We illustrate the performance of emulation and history matching on a simple one-dimensional toy model and in application to a climate model.This article is part of the theme issue ‘Reliability and reproducibility in computational science: implementing verification, validation and uncertainty quantification in silico’.

诸多计算机模型拥有高维输入空间，且单次模型评估需耗费大量计算时间。尽管此类模型通常为确定性模型，但仍存在多种不确定性。我们认为，不确定性量化（uncertainty quantification）对于计算机模型的验证与可重复性至关重要。我们提出了一种名为历史匹配（history matching）的统计框架，通过将模型输出与观测数据进行对比，实现全局参数搜索。我们采用高斯过程（Gaussian process, GP）模拟器，可对任意输入参数设置下的模型行为进行快速预测，进而计算输出的不确定性分布。基于我们对不确定性的表征方式，历史匹配可识别出可使观测数据与模型输出达到可接受匹配程度的输入参数集合。建模人员可在这些识别出的参数设置下，运行计算机模型以获取目标输出，并生成一系列预测结果。模型结果的变异性对于模型间对比以及模型开发均具有关键意义。我们通过一个简单的一维玩具模型以及某气候模型的应用案例，展示了模拟器与历史匹配方法的性能表现。本文属于专题专栏《计算科学中的可靠性与可重复性：在计算机模拟（in silico）中实现验证、确认与不确定性量化》的一部分。