Metatasks for AutoGluon - ROC AUC and Balanced Accuracy

Prediction Data of Base Models from AutoGluon on 71 classification datasets from the AutoML Benchmark for Balanced Accuracy and ROC AUC. <br> The files of this figshare item include data that was collected for the paper: <strong>CMA-ES for Post Hoc Ensembling in AutoML: A Great Success and Salvageable Failure</strong>, <em>Lennart Purucker, Joeran Beel, Second</em> <em>International Conference on Automated Machine Learning, 2023.</em> <br> The data was stored and used with the <em>assembled </em>framework: https://github.com/ISG-Siegen/assembled. <br> In detail, the data contains the predictions of base models on validation and test as produced by running AutoGluon for 4 hours. Such prediction data is included for each model produced by AutoGluon on each fold of 10-fold cross-validation on the 71 classification datasets from the AutoML Benchmark. The data exists for two metrics (ROC AUC and Balanced Accuracy). More details can be found in the paper. <br> The data was collected by code created for the paper and is available in its <em>reproducibility repository</em>: https://doi.org/10.6084/m9.figshare.23609226. <br> Its usage is intended for but not limited to using <em>assembled </em>to evaluate post hoc ensembling methods for AutoML.<br> Details The link above points to a hosted server that facilitates the download. We opted for a hosted server, as we found no other suitable solution to share these large files (due to file size or storage limits) for a reasonable price. If you want to obtain the data in another way or know of a more suitable alternative, please contact Lennart Purucker. <br> The link resolves to a directory containing the following: example_metatasks: contains an example metatask for test purposes before committing to downloading all files. metatasks_roc_auc.zip: The Metatasks obtained by running AutoGluon for ROC AUC. metatasks_bacc.zip: The Metatasks obtained by running AutoGluon for Balanced Accuracy. <br> The size after unzipping is: metatasks_roc_auc.zip: ~85GB metatasks_bacc.zip: ~100GB <br> The metatask .zip files contain 2 files per metatask. One .json file with metadata information and a .hdf file containing the prediction data. The details on how this should be read and used as a Metatask can be found in the <em>assembled </em>framework and the reproducibility repository. To obtain the data without Metataks, we advise looking at the file content and metadata individually or parsing them by using Metatasks first.