MBT4CPS: Evolving, Modelling and Testing Realistic and Risky Uncertain Behaviours of Cyber-Physical Systems, 2018

The data are evaluation results for a executable model-based testing approach, which contains three parts. First, a set of test models were created to evaluate the applicability of a modelling framework (MoSH) for the testing approach. This corresponds to the models.zip file. Second, the performance of a testing framework (TM-Executor) was evaluated. The files (enter state.csv, evaluate constraint.csv, execute operation.csv, exit state.csv, generate test input.csv, introduce uncertainty.csv, traverse transition.csv) store the time cost of the testing framework to perform the test actions. Third, the file "testing results.csv" contains the number of faults detected by two operation invocation strategies and two uncertainty introducing strategies, for the nine behaviours of three systems. The traces.zip file contains detailed testing traces. For the test models, systems' names were used as the names of the models. Regarding the time performance data, the time costs of TM-Executor were saved in CSV files, including "enter state.csv", "exit state.csv", "traverse transition.csv", "execute operation.csv", "generate test input.csv", "evaluate constraint.csv", "introduce uncertainty.csv". The first four files only contain one column, which captures the time spent by TM-Executor to perform each action in nanoseconds. "generate test input.csv" and "evaluate constraint.csv" contains three column: number of clauses, parameter types, and time cost in nanoseconds. "introduce uncertainty.csv" also has three column: uncertainty level, uncertainty measure, time cost in nanoseconds. Concerning the testing results data, the numbers of detected faults were saved in a single CSV file, named as "testing results.csv". The file contains 47 columns. The first three columns are testing budget in hours, system name, and self-healing behavior name. The remaining 44 columns are the number of faults detected by four testing approaches, i.e., FOT+UPO, FOT+RANDOM, COT+UPO, COT+RANDOM, in 10 runs of the experiment and the sum of detected faults in the 10 runs. For testing traces, systems' names, strategies' names, and the timestamps of the starting time points of the execution as the names of trace files were used.本数据集为基于可执行模型的测试方法的评估结果，共包含三部分内容。第一部分为一组测试模型，用于评估某建模框架（MoSH）对该测试方法的适配性，对应文件为models.zip。第二部分为对某测试框架（TM-Executor）的性能评估，相关文件包括enter state.csv、evaluate constraint.csv、execute operation.csv、exit state.csv、generate test input.csv、introduce uncertainty.csv、traverse transition.csv，这些文件存储了该测试框架执行各类测试操作时的时间开销。第三部分为testing results.csv文件，其中记录了针对三个系统的九种行为，两种操作调用策略与两种不确定性引入策略所检测到的故障数量。traces.zip文件则包含了详细的测试轨迹。 对于测试模型，其命名采用对应系统的名称。在时间性能数据方面，TM-Executor的时间开销存储于多个CSV文件中，包括enter state.csv、exit state.csv、traverse transition.csv、execute operation.csv、generate test input.csv、evaluate constraint.csv、introduce uncertainty.csv。其中前四个文件仅包含单列数据，用于记录TM-Executor执行单次操作所花费的纳秒级时间。generate test input.csv与evaluate constraint.csv包含三列数据，分别为子句数量、参数类型以及纳秒级时间开销。introduce uncertainty.csv同样包含三列：不确定性等级、不确定性度量值以及纳秒级时间开销。关于测试结果数据，检测到的故障数量存储于名为testing results.csv的单个CSV文件中，该文件共包含47列。前三列分别为测试时长预算（单位：小时）、系统名称以及自修复行为名称，剩余44列则对应四种测试方法（即FOT+UPO、FOT+RANDOM、COT+UPO、COT+RANDOM）在10次实验运行中检测到的故障数量，以及这10次运行中检测到的故障总数。对于测试轨迹文件，其命名采用系统名称、策略名称以及执行起始时间戳的组合形式。