Theory-Guided Online Algorithm Selection: Benchmarking Algorithm Switching for Pseudo-Boolean Optimization - Reproducibility Files

# Reproducibility files and additional results ## Additional Material The addional results mentioned in the paper can be found in the highlighted pdf file. ## Content: - data_collection: contains all algorithms and code to collect performance data on the selected benchmarks. In the main function, there are a set of experiments to collect data, each of which should be run if all experiments from the paper should be reproduced. - Data_raw.zip: Split up in the same way as the experiments in data_collection.py, this contains the IOH-logs from the different algorithm-benchmark combinations. - Visualize.ipynb: Notebook which takes the performance data and creates all figures (both the ones in the paper + the additional figures provided here) - Figures.zip: The full set of figures generated - Supplementary_materials.pdf: The pdf with the figures and additional explanations on results which did not fit into the main body of the paper. # Reproducibility instructions ## Data collection For the data collection, the only required package is iohexperimenter (ioh on pip). Then, the data_collection script can be run to get the data for a given experiment by modifying the used functions, algorithms and dimensionalities. The code has 4 interfaces for running algorithms: - run_algorithm: works for the base algorithms with flexible population size. Argument order [algname, function_id, dim, lambda_, override] - run_algorithm_switch: works for the switching algorithm with flexible population size and stagnation threshold. Argument order [algname, function_id, dim, lambda_, stagnation_threshold, override] - run_alg_fixedpop: same as run_algorithm, but without the population size variability - run_sw_alg_fixedpop: same as run_algorithm_switch, but without the population size variability The data provided is split into 5 separate folders for different parts of the experiments (separating the base data for stagnation threshold, the SAT problems and the fixed population size experiments) ## Visualization Self-contained notebook. Can be run on the provided data or the rerun data from data_Collection.py. This notebook relies on the iohinspector package for data processing, and on seaborn for plotting.