Benchmarking Dynamic Binary Value Problems with IOHprofiler -- Reproducibility files

This repository contains the code, data and scripts to reproduce the results from the paper "Empirical Analysis of the Dynamic Binary Value Problem with IOHprofiler".

The files are split into 3 parts:

Part 1: Data collection

The data can be collected using the data_collection.py file. In its default state, this will collect the full set of experimental data used in the paper. This can be modified by adjusting the main to only execute the relevant parts of the experiments. Note that the data location and number of parallell processes can be adjusted at the top of this file.

To run the data collection, the only requirement is to install multiprocessing (any version from pip) and iohexperimenter. The version of ioh should be installed from github https://github.com/IOHprofiler/IOHexperimenter/tree/dynamic-bin-val (installation instructions can be found on the github page). Alternatively, the future pip-release 0.3.16 will include these changes as well.

Part 2: Data

Instead of rerunning the full experimental data collection, the results of this have also been made available. Each set of experiments is provided as a 'SetX.zip' file.

Part 3: Visualization

To analyze and visualize the data, the provided notebook can be used. This notebook takes the data from either step 1 or 2 and uses it to create the figures presented in the paper (plus some variations).

The resulting figures are also provided on Figshare: doi.org/10.6084/m9.figshare.25592904