Replication or Exploration? Sequential Design for Stochastic Simulation Experiments

We investigate the merits of replication, and provide methods for optimal design (including replicates), with the goal of obtaining globally accurate emulation of <i>noisy</i> computer simulation experiments. We first show that replication can be beneficial from both design and computational perspectives, in the context of Gaussian process surrogate modeling. We then develop a lookahead-based sequential design scheme that can determine if a new run should be at an existing input location (i.e., replicate) or at a new one (explore). When paired with a newly developed heteroscedastic Gaussian process model, our dynamic design scheme facilitates learning of signal and noise relationships which can vary throughout the input space. We show that it does so efficiently, on both computational and statistical grounds. In addition to illustrative synthetic examples, we demonstrate performance on two challenging real-data simulation experiments, from inventory management and epidemiology. Supplementary materials for the article are available online.

本研究探讨了重复实验的优势，并提出了包含重复实验的最优设计方法，旨在实现带噪声（noisy）计算机仿真实验的全局精准模拟。首先，在高斯过程代理建模（Gaussian process surrogate modeling）框架下，我们证明了重复实验在设计与计算层面均具备优势。随后，我们提出了一种基于前瞻策略的序贯设计方案，可判定新实验运行点应选取在已有输入位置（即重复实验）还是全新的输入位置（即探索新区域）。当与新近提出的异方差高斯过程模型结合使用时，本动态设计方案可助力学习输入空间内各处存在差异的信号与噪声关系。研究表明，该方案在计算与统计层面均能实现高效运行。除了带有演示性质的合成示例外，我们还在两个极具挑战性的真实数据仿真实验（分别来自库存管理与流行病学领域）中验证了方案的性能。本文的补充材料可在线获取。