experimental data (raw data, processed data)

##################################################################################  Source code and problem instances:###############################################################################- The source code of METAFOR will be made available on github once the paper has been accepted for publication. In the meantime, the code is provided in METAFOR.zip- The list of problem instances (single objective continuous functions) are provided in the attached file "instances.zip"###############################################################################

################################################################################## Experiment folders:###############################################################################The folders "default", "leave25OUT", "leave25OUTCEC14", "leaveLDO" and "leaveLDOCEC14" (compressed in zip format to save space) contain all the data collected during the experiment. Each of them contains the following folders/files:     - "candidates/candidates.txt" -- a text file with the algorithms specified as command lines,     - "candidates/OUTPUT" -- a folder containing a text file with the best solutions found by each algorithm for each problem instance. The names of the files in the folder are composed of the test suite "c_<0,1,2,3>", the number of the function in the test suite "f_<0,...,n>", and the number of dimensions "d_<50,100,500,750,...,d>".     - "DataAndPlots" -- a folder created automatically by the "plot_bxp_rtd_wlx.sh" processing script (see below).         Inside this folder are the following subfolders:          - "DataAndPlots/Bxp" -- stores the box plots created based on the data in "DataAndPlots/Data";          - "DataAndPlots/Cvg" -- (if any)stores the convergence plots generated based on the data in "OUTPUT_processed";          - "DataAndPlots/Data" -- stores the processed data and statistical information (median, median error, statistical test, etc.) of the raw data stored in "candidates/OUTPUT";          - "DataAndPlots/Time" -- stores the average time taken by the algorithms on each problem instance.          - "DataAndPlots/Table" -- (if any) stores, in plain text and pseudo-LaTeX format, the tables of results reported in the paper, i.e., median, median error, median absolute deviation, rankings, statistical tests, and number of wins.     - "OUTPUT" -- stores the convergence data of the algorithms (i.e. function evaluations vs. solution quality).        In the latest version of METAFOR, each convergence file consists of 100 points; however, in the version we used for the experiments reported in the paper, each file consists of thousands of points per algorithm, making this folder particularly heavy.     The data in folder "candidates/OUTPUT" and "OUTPUT" is gathered via the script "runMe.sh" indicating an experiment folder and instances file, namely:- in the case of "default", we solved instances "test_MIXTURE_max200.txt" and "test_MIXTURE_onlyLargeScale.txt";- in the case of "leave25OUT", we solved instances "test_MIXTURE_max200.txt", "test_MIXTURE_onlyLargeScale.txt" and "test_MIXTURE_onlyLargeScaleDND.txt";- in the case of "leave25OUTCEC14", we solved instances "test_CEC14.txt";- in the case of "leaveLDO", we solved instances "test_MIXTURE_max200.txt", "test_MIXTURE_onlyLargeScale.txt" and "test_MIXTURE_onlyLargeScaleDND.txt";- in the case of "leaveLDOCEC14", we solved instances "test_CEC14.txt".###############################################################################

################################################################################## Processing scripts:###############################################################################The scripts folder (scripts.zip) contains the main processing script "plot_bxp_rtd_wlx.sh" and several auxiliary R and shell scripts: "boxplot.R", "cvg_log.R", "wilcoxon.R", "ranksPerClass.R", "filter_repeating.sh", "full_outer_join.sh", and "replace_na.sh". All the auxiliary scripts are automatically called by the main script, depending on the options specified by the user, and are intended to be used standalone. The R axiliary scripts are used to generate the box plots ("boxplot.R") and convergence plots ("cvg_log.R"), to perform the statistical test ("wilcoxon.R"), and to compute the rankings ("ranksPerClass.R"). The auxiliary shell scripts are used to clean the raw data stored in "OUTPUT" and create a file called data-*-mean.txt for each data file, which can be entered into "cvg_log.R" to generate the convergence plots.###############################################################################

################################################################################## Folders with experiments:###############################################################################Since in the paper we report different sets of algorithms solving different sets of problems. We created a folder (compressed in zip format for space reasons) for each of them and put in it only the specific data that we want to analyze and plot. The data inside these folders is simply copy-paste from the main experiment folders, and it is as follows:- "METAFOR/exp1_dftVStuned" contains the results discussed in section 5.3.1 of the paper.- "METAFOR/exp2_mtfVSHyb" contains the results discussed in section 5.3.2 of the paper.- "METAFOR/exp3_CEC14" and "METAFOR/exp4_LS" contain the results discussed in section 5.3.3 of the paper.###############################################################################