MedMNIST-NAS-Bench: A Tabular Neural Architecture Search Benchmark on MedMNIST v2

MedMNIST-NAS-Bench is a tabular neural architecture search (NAS) benchmark covering four MedMNIST v2 medical-imaging datasets (BreastMNIST, PneumoniaMNIST, DermaMNIST, and BloodMNIST) on the native 28×28 image resolution. The benchmark is released across two search spaces of different cardinality: tiny: a focused space of 768 architectures spanning three macro families (MobileNet, ResNet, and the Hybrid HYB_RES_MB), squeeze-and-excitation attention, and strided-convolution downsampling. small: an extended space of 5,016 architectures that additionally includes ConvNeXt-style and HYB_RES_NEXT templates, bottleneck channel growth, ECA attention, patch-merge downsampling, and a wider range of width multipliers. For each (dataset, search space, architecture) combination, training is run from scratch for 100 epochs under three random seeds (777, 888, 999), producing 9,216 fully trained runs in the tiny space and 60,192 runs in the small space, for a total of 69,408 runs across the benchmark. Each run records 60+ per-epoch metrics (accuracy, AUROC, F1, ECE, calibration error, loss, gradient-norm aggregates, learning rate, timing, and resource use) and validation/test logits at the final and best-validation epochs, enabling downstream users to (i) rank architectures by any standard metric, (ii) recompute custom metrics from saved predictions, and (iii) study optimisation dynamics. Data ships in both HDF5 (predictions + labels + metric arrays) and Apache Parquet (long-format tidy table) formats. Loader code, schema documentation, and the exact build script are available at https://github.com/jiywww/medmnist-nas-bench under the Apache-2.0 licence. The MedMNIST labels redistributed in the HDF5 files are reused from MedMNIST v2 (Yang et al. 2023) under CC BY 4.0.