Surrogate Modeling Benchmark - Two-dimensional heat diffusion model

This dataset is related to the Two-dimensional heat diffusion model benchmark case. A detailed description of the benchmark case can be found on the public online community website UQWorld: https://uqworld.org/t/benchmark-case-two-dimensional-heat-diffusion-model/. The experimental designs include datasets with 400, 800, 1200, 1600, and 2000 samples, each generated using optimized maximin distance Latin Hypercube Sampling (LHS) with 1000 iterations. Each dataset is replicated 20 times. The validation set contains 100,000 samples generated by Monte Carlo simulation. Each dataset contains input samples and the corresponding computational model responses. Description of the dataset file The dataset file includes two variables: ExpDesigns, and ValidationSet. Both variables are Matlab structures with fields X, Y, and nSamples. Variable ExpDesigns is a non-scalar structure sized according to the number of experimental design groups. Each field of X for the i-th element of the struct array contains replicated datasets, forming a matrix of size [number of samples] x [dimensionality] x [number of replications]. Similarly, each field of Y for the i-th element contains replicated computational model responses that correspond to the experimental design of the same replication, sized [number of samples] x [number of model outputs] x [number of replications]. The same structure logic applies to the ValidationSet variable, except it contains only one dataset per benchmark case. The structure can be summarized as follows: ExpDesigns(i).X(j,k,l) i: dataset group, j: sample index, k: variable index, and l: replication index. ExpDesigns(i).Y(j,m,l) i, j, l: same as above, m: computational model output index. ValidationSet.X(j,k) j, k: same as above. ValidationSet.Y(j,m) j, m: same as above. Description of benchmarked metamodel competitors The selection of competitors was based on our experience with meta-modeling and includes various metamodel types: Polynomial Chaos Expansions (PCE), Polynomial Chaos Kriging (PCK), and Kriging. Given that each metamodel has many hyperparameters, we chose the most general settings to address different benchmark case difficulties, including dimensionality, nonlinearity, and non-monotonicity. For Polynomial Chaos Expansions (PCE), we used a polynomial degree and q-norm adaptivity approach. This approach adaptively increases the maximum polynomial degree and truncation q-norm until the estimated leave-one-out error starts increasing. Maximum polynomial interaction terms were limited to 2 due to the memory requirements for large model dimensionality and large experimental designs. We tested three different solvers to calculate the PCE coefficients: Least Angle Regression (LARS), Orthogonal Matching Pursuit (OMP), and Subspace Pursuit (SP). Polynomial Chaos Kriging (PCK) employs a sequential combination strategy of PCE and Kriging. PCE uses degree adaptivity with a fixed q-norm. The maximum number of interactions is again set to 2 with the LARS solver. Ordinary Kriging is applied using the Matérn-5/2 correlation family, ellipsoidal, and anisotropic correlation function. We used a hybrid genetic algorithm to optimize the hyperparameters. We benchmarked both linear and ordinary Kriging, including Matérn-5/2 and Gaussian correlation families and separable and ellipsoidal correlation, resulting in eight different Kriging competitors. The hyperparameters were calculated using a hybrid covariance matrix adaptation-evolution strategy optimization. For further details on the settings, please refer to the competitors.m file and UQLab user manuals: S. Marelli, N. Luethen, B. Sudret, UQLab User Manual – Polynomial Chaos Expansions, Report UQLab-V2.1-104, Chair of Risk, Safety and Uncertainty Quantification, ETH Zurich, Switzerland, 2024. C. Lataniotis, D. Wicaksono, S. Marelli, B. Sudret, UQLab User Manual – Kriging (Gaussian Process Modeling), Report UQLab-V2.1-105, Chair of Risk, Safety and Uncertainty Quantification, ETH Zurich, Switzerland, 2024. R. Schoebi, S. Marelli, B. Sudret, UQLab User Manual – Polynomial Chaos Kriging, Report UQLab-V2.0-109, Chair of Risk, Safety and Uncertainty Quantification, ETH Zurich, Switzerland, 2022. Description of the results file The results file contains one variable: Metrics. It is a Matlab structure with fields corresponding to each competitor (currently 12). Each competitor field contains data of type non-scalar struct array. The performance metrics included are RelMSE, RelRMSE, RelMAE, MAPE, Q2, and RelCVErr. Each field of Metrics.(CompetitorName) for the i-th element of the struct array contains metrics corresponding to the replicated dataset and the competitor, structured as follows: Metrics.(CompetitorName)(i).(MetricName)(l) i: dataset group, l: replication index. The description of the performance measures (metrics) can be found here: https://uqworld.org/t/metamodel-performance-measures/. Additional files We provide files in three languages (MATLAB, Python, and Julia) to showcase how to work with datasets, results, and their visualization. The files are called working_with_datafiles.* (the extension depends on the selected language). Acknowledgment This project was supported by the Open Research Data Program of the ETH Board under Grant number EPFL SCR0902285. The calculations were run on the Euler cluster of ETH Zürich using the MATLAB-based UQLab software developed at the Chair of Risk, Safety and Uncertainty Quantification of ETH Zürich.本数据集与二维热扩散模型基准算例相关。该基准算例的详细说明可在公开在线社区网站UQWorld查阅，链接为：https://uqworld.org/t/benchmark-case-two-dimensional-heat-diffusion-model/。 实验设计涵盖样本量分别为400、800、1200、1600及2000的数据集，所有数据集均采用经1000次迭代优化的最大最小距离拉丁超立方抽样（Latin Hypercube Sampling, LHS）生成，且每组数据集均重复20次。验证集包含100000个通过蒙特卡洛模拟（Monte Carlo simulation）生成的样本。所有数据集均包含输入样本与对应的计算模型响应。 数据集文件说明 本数据集文件包含两个变量：`ExpDesigns`与`ValidationSet`。二者均为MATLAB结构体，包含字段`X`、`Y`与`nSamples`。其中`ExpDesigns`为非标量结构体，其尺寸与实验设计组的数量对应。结构体数组第`i`个元素的`X`字段存储重复数据集，形成尺寸为`[样本数] × [维度数] × [重复次数]`的矩阵。同理，第`i`个元素的`Y`字段存储与对应重复实验设计匹配的重复计算模型响应，尺寸为`[样本数] × [模型输出数] × [重复次数]`。`ValidationSet`变量遵循相同的结构体逻辑，仅针对单个基准算例包含一组数据集。 该结构体的具体结构如下： 1. `ExpDesigns(i).X(j,k,l)`，其中： - `i`：数据集组别 - `j`：样本索引 - `k`：变量索引 - `l`：重复索引 2. `ExpDesigns(i).Y(j,m,l)`，其中`i`、`j`、`l`含义同上，`m`为计算模型输出索引。 3. `ValidationSet.X(j,k)`，其中`j`、`k`含义同上。 4. `ValidationSet.Y(j,m)`，其中`j`、`m`含义同上。 基准元模型竞品说明 竞品的选取基于我们在元建模领域的实践经验，涵盖多种元模型类型：多项式混沌展开（Polynomial Chaos Expansions, PCE）、多项式混沌克里金（Polynomial Chaos Kriging, PCK）以及克里金（Kriging）。鉴于每种元模型均包含大量超参数，我们选择最通用的设置以适配不同基准算例的难度，包括维度、非线性与非单调性。 对于多项式混沌展开（PCE），我们采用多项式阶数与q范数自适应方法：该方法会自适应提升最大多项式阶数与截断q范数，直至估计的留一法误差开始上升。考虑到大模型维度与大实验设计的内存需求，最大多项式交互项被限制为2。我们测试了三种用于计算PCE系数的求解器：最小角回归（Least Angle Regression, LARS）、正交匹配追踪（Orthogonal Matching Pursuit, OMP）以及子空间追踪（Subspace Pursuit, SP）。 多项式混沌克里金（PCK）采用PCE与克里金的顺序组合策略。PCE采用固定q范数的阶数自适应方法，最大交互项数同样设置为2，并使用LARS求解器。普通克里金采用Matérn-5/2相关核族、椭球型各向异性相关函数。我们使用混合遗传算法优化超参数。 我们对线性克里金与普通克里金均进行了基准测试，涵盖Matérn-5/2与高斯相关核族，以及可分离与椭球型相关函数，共得到8种不同的克里金竞品。超参数通过混合协方差矩阵自适应进化策略（covariance matrix adaptation-evolution strategy, CMA-ES）优化计算。 如需了解设置的更多细节，请参阅`competitors.m`文件以及UQLab用户手册： S. Marelli, N. Luethen, B. Sudret, UQLab User Manual – Polynomial Chaos Expansions, Report UQLab-V2.1-104, Chair of Risk, Safety and Uncertainty Quantification, 瑞士苏黎世联邦理工学院, 2024. C. Lataniotis, D. Wicaksono, S. Marelli, B. Sudret, UQLab User Manual – Kriging (Gaussian Process Modeling), Report UQLab-V2.1-105, Chair of Risk, Safety and Uncertainty Quantification, 瑞士苏黎世联邦理工学院, 2024. R. Schoebi, S. Marelli, B. Sudret, UQLab User Manual – Polynomial Chaos Kriging, Report UQLab-V2.0-109, Chair of Risk, Safety and Uncertainty Quantification, 瑞士苏黎世联邦理工学院, 2022. 结果文件说明 结果文件包含一个变量：`Metrics`。该变量为MATLAB结构体，其字段与每种竞品对应（目前共12种）。每个竞品字段均为非标量结构体数组。包含的性能指标有相对均方误差（RelMSE）、相对均方根误差（RelRMSE）、相对平均绝对误差（RelMAE）、平均绝对百分比误差（MAPE）、决定系数（Q2）以及相对交叉验证误差（RelCVErr）。结构体数组第`i`个元素的`Metrics.(CompetitorName)`字段中，每个指标字段存储对应数据集组别与竞品的重复数据集性能指标，具体结构如下： `Metrics.(CompetitorName)(i).(MetricName)(l)`，其中： - `i`：数据集组别 - `l`：重复索引 性能指标的详细说明可参阅：https://uqworld.org/t/metamodel-performance-measures/ 附加文件 我们提供了三种语言（MATLAB、Python与Julia）的示例文件，用于演示如何处理数据集、结果文件以及进行可视化。这些文件命名为`working_with_datafiles.*`，文件扩展名取决于所选的编程语言。 致谢 本项目获得苏黎世联邦理工学院董事会开放研究数据计划（资助编号EPFL SCR0902285）的资助。计算工作在苏黎世联邦理工学院的欧拉集群（Euler cluster）上完成，使用了由瑞士苏黎世联邦理工学院风险、安全与不确定性量化教研室开发的基于MATLAB的UQLab软件。