Data, Results, and Figures for Specialising Algorithm Selectors Over Time for Recurring Problems by Classifying Best-so-far Performance Trajectories

Data and additional figures associated with the paper "Specialising Algorithm Selectors Over Time for Recurring Problems by Classifying Best-so-far Performance Trajectories". The following .zip files are included: traj_seed30.zip - This contains trajectory data for 24 BBOB problems, for 25 instances each, for dimensions 2, 10, and 100 for 10000 evaluations. output_all_algos.zip - This contains accuracy, performance, and plotting output for experiments on 34 algorithms. output_6_algos.zip - This contains accuracy, performance, and plotting output for experiments on 6 algorithms. Both .zip files with experiment output are structured as follows: accuracy - Directory with classifier output probabilities and accuracy measures per cross-validation fold, problem, dimensionality, budget. performance - Directory with raw selector performances values and algorithm choice on test instances per method, cross-validation fold, iteration, dimensionality, budget plots - Directory with the following plots: accuracy - Plots showing accuracy based on various measures, the paper discusses: True problem - Accuracy with which the true problem was predicted. Both per fold, and aggregated for combinations of dimensionality and budget. All algorithm match - Accuracy with which a problem was predicted for which all algorithms in the single best solver match with those for the true problem. Both per fold, and aggregated for combinations of dimensionality and budget. loss - Loss of the adaptive selector to either the static selector or the VBS aggregate - Various aggregations over all test instances and folds. E.g., over all problems, all budgets, all dimensionalities, or over specific dimensionality-budget combinations. aggregate_no_draws - Various aggregations over the test instances for which the two methods did not have the same performance (i.e., draw) and folds. E.g., over all problems, all budgets, all dimensionalities, or over specific dimensionality-budget combinations. scatter - Scatterplots showing the raw performance of the adaptive selector compared to either the static selector or the VBS. Aggregated over all scenarios, and over individual dimensionality-budget combinations. selectors - The selectors as constructed during the experiments. Selectors themselves in .csv, settings used to construct the selectors in .yaml. win_counts.txt - The number of wins, draws, and losses of the adaptive selector compared to the static selector and the VBS, per dimensionality and budget combination.