On the Influence of the Feature Computation Budget on Per-Instance Algorithm Selection for Black-Box Optimisation - Reproducibility Files

# Reproducibility and Additional Data This repository contains ## Structure: The following files are provided: - base.py: Contains the code for function definitions and some utilities (including datapath for data generation) - get_ELA.py: Code for calculating ELA features - get_performance.py: Code for running algorithms on the problems and recording performance data - Data_Raw: raw performance data (IOH-format) - ELA_Raw.zip: raw ELA data (before processing) - Processing.ipynb: Code to process the performance and ELA data into usable formats for algorithm selection - Data.zip: Processed performance data - ELA.zip: Processed ELA data - Selection.zip: Code to run the algorithm selection pipeline on all scenarios - Visualization.ipynb: Notebook to visualize the selection results - Figures.zip: All generated figures from the visualization notebook (figs from the paper + additional versions not included for space reasons) Some additional files: - ROG: the exact functions generated using RandOptGen - requirements: the requirements for running the data collection # Reproducibility instructions ## Data generation: To generate the raw data, run the get_* files. It is recommended to do so on a server, as it can take some time to run all algorithms. The functions in base.py provide some parallelization, this can be adapted to fit to the used machine. Note that file-paths should also be set before running. ## Data Processing: To go from the raw data to data which works well with algorithm selection, we need to do the following: - Exctract the fixed-budget performance and transfer it to the normalized metric (attainment or normalized function value) - Normalize the ELA feature vectors ## Algortihm selection: For the algorithm selection, refer to the code in 'Selection.zip' ## Visualization: For generating the figures from the selection, we use the Visualize.ipynb file. This notebook contains the steps required to create each figure type from the paper. ## Questions In case of questions, don't hesitate to reach out to any of the authors.