MBT4CPS: Evolving, Modelling and Testing Realistic and Risky Uncertain Behaviours of Cyber-Physical Systems, 2018

The data are evaluation results for a executable model-based testing approach, which contains three parts. First, a set of test models were created to evaluate the applicability of a modelling framework (MoSH) for the testing approach. This corresponds to the models.zip file. Second, the performance of a testing framework (TM-Executor) was evaluated. The files (enter state.csv, evaluate constraint.csv, execute operation.csv, exit state.csv, generate test input.csv, introduce uncertainty.csv, traverse transition.csv) store the time cost of the testing framework to perform the test actions. Third, the file "testing results.csv" contains the number of faults detected by two operation invocation strategies and two uncertainty introducing strategies, for the nine behaviours of three systems. The traces.zip file contains detailed testing traces.

For the test models, systems' names were used as the names of the models. Regarding the time performance data, the time costs of TM-Executor were saved in CSV files, including "enter state.csv", "exit state.csv", "traverse transition.csv", "execute operation.csv", "generate test input.csv", "evaluate constraint.csv", "introduce uncertainty.csv". The first four files only contain one column, which captures the time spent by TM-Executor to perform each action in nanoseconds. "generate test input.csv" and "evaluate constraint.csv" contains three column: number of clauses, parameter types, and time cost in nanoseconds. "introduce uncertainty.csv" also has three column: uncertainty level, uncertainty measure, time cost in nanoseconds. Concerning the testing results data, the numbers of detected faults were saved in a single CSV file, named as "testing results.csv". The file contains 47 columns. The first three columns are testing budget in hours, system name, and self-healing behavior name. The remaining 44 columns are the number of faults detected by four testing approaches, i.e., FOT+UPO, FOT+RANDOM, COT+UPO, COT+RANDOM, in 10 runs of the experiment and the sum of detected faults in the 10 runs. For testing traces, systems' names, strategies' names, and the timestamps of the starting time points of the execution as the names of trace files were used.