Replication Package for: Automatic Parameter Optimization for Stream Processing Systems

This replication package contains the data and scripts to replicate the results of the master thesis "Automatic Parameter Optimization for Stream Processing Systems". All listed results were obtained by executed the configurations on 3x CloudLab m510 instances. Dataset Description The results of the parOpt executions are located at the root directory, grouped by their phases. In same cases the phases were executed over multiple reprovisions, which is reflected in the folder structure (part1, part2).In addition the rerun_best_hc contains the repetition of the best performing HC iterations, and sa_latency_results the results for the top 50% of SA iterations with a 80% load of the achieved throughput for latency measurements. The files are named following way: exp0_{LOAD}_{INSTANCES}_generic_throughput_{IDX}.csv where: {LOAD} is the number generated per second {INSTANCES} is the number of used instances (3 in all cases) {IDX} is the number of the repetition of an execution (1-5). Each of these CVS files contains the throughput measurements of the benchmark executions, every 5 seconds. The important columns are: timestamp in epoch seconds, value the measured throughput in records per second as obtained with the ad-hoc throughput metric of ShuffleBench. Each folder also contains a execution folder, which contains the configuration yaml files. These can be used to replicate the the experiments. Analysis To repeat our data analysis, you can use the evaluator.py following these steps: Install python 3.10 Install the required Python packages:pip install pandas json os re Run the file python3 evaluator.py The evaluator prints the execution name, parameters and the mean + standard deviation columns for a given results folder. It can be configured to also write the result to a file.