Latency comparison between baseline execution and TEE enclave execution of a multi-model consistency checker

Content This dataset contains the raw latency measurements and supplementary artifacts used in the evaluation of a Trusted Execution Environment (TEE)-based multi-model consistency checker, as presented in the accompanying research paper. The dataset was collected to quantify the runtime overhead introduced by executing a cross-organizational consistency checker inside an Intel SGX enclave compared to native execution. The measurements were obtained by repeatedly executing consistency checking scenarios under two configurations: (1) baseline execution without a TEE and (2) execution inside an SGX enclave using the Occlum LibOS. The scenarios vary in the number of checked model elements, the number of data requests issued by the checker, and the number of performed rule evaluations. Both single-element checks and recursive checks over larger model structures were evaluated. The dataset contains timestamp-derived latency measurements for each individual run, along with metadata describing the corresponding scenario, including trigger component, check type (single or recursive), presence of simulated inconsistencies, number of checked elements, number of data requests, and number of rule evaluations. Changes from v1 to v2 Compared to the first dataset release, this second version extends the measurement campaign with an additional scaled-up consistency checking scenario intended to provide an initial scalability insight. While the original fourteen scenarios remain unchanged and are included with all previously published measurements (200 iterations per scenario, making it 2,800 in total), the new scenario contributes 30 additional executions (15 native executions and 15 TEE-based executions). As a result, the dataset now comprises 2,830 individual latency measurements instead of 2,800. In addition to the raw latency values, the dataset now also provides supplementary statistical measures, including the interquartile range (IQR) and relative standard deviation (%RSD), to support a more detailed assessment of runtime variability. File description The primary dataset file is: TEE_consistency_checking_performance_comparison.xlsx: Contains all raw latency measurements from both dataset versions, including the newly added scalability scenario and the additional statistical analyses. To improve transparency and support future replication efforts, this version further includes the following supplementary artifacts. While these artifacts were already part of the experimental setup used to generate the measurements, they were not included in the initial dataset release and are now provided to improve transparency and reproducibility. rules.xml: The consistency rule set used by the consistency checker during all evaluated scenarios. links.xml: The model element links defining the cross-model relationships considered during consistency checking. test_models.zip: Contains all engineering models used in the experiments, including: Two OpenModelica model files (.mo), Three representations of the CAD model "combined_upper_rack" (.f3d, .step, and a .zip archive containing Autodesk Inventor .iam and .ipt files), Two AutoFOCUS3 model files (.af3_23), consisting of the original model and the scaled-up model variant used for the scalability scenario. viewtype_collection.zip: Contains the viewtype collection used by the central consistency checker, including the corresponding Ecore and GenModel definitions. Usage The provided measurements, models, consistency rules, model links, and viewtype definitions enable a detailed understanding of the evaluated consistency checking scenarios and support independent analysis of the reported results. Reproducing the measurements with identical latency values would require access to a comparable hardware and software environment, including Intel SGX-capable hardware and the same TEE execution stack. While exact replication is therefore unlikely, similar latency overhead characteristics are expected when executing the consistency checker in comparable TEE environments. The experimental setup described in the accompanying paper executes the complete Java-based central consistency checker C4 inside an Intel SGX enclave using the Occlum LibOS. Although C4 is available as open-source software, the framework and its tool-specific adapters are still under active development. At the time of publication, no comprehensive setup guide is available for the complete environment, including the integrations with AutoFOCUS3, OpenModelica, and Autodesk Fusion 360. Consequently, detailed deployment instructions are not provided as part of this dataset. Researchers interested in trying out the C4 are encouraged to contact the MBSE team at fortiss ([e-mail]). For understanding the evaluated scenarios, analyzing the measurements, and reviewing the employed consistency checking artifacts, the material contained in this dataset is sufficient.