Beyond Point Estimates: Data and figure reproduction for the confidence-interval evaluation of ML performance metrics

Beyond Point Estimates — data and figure reproduction This record accompanies the paper "Beyond Point Estimates: Distributional Uncertainty in Machine Learning Performance Evaluation" and the code repository BeyondPoint-experiments. It contains the static results needed to reproduce the paper figures, the individually estimated confidence intervals, and the empirical performance distributions of the underlying experiments (three use cases: CIFAR-10 classification, Simpsons classification, and Superconductor regression). The full end-to-end evaluation pipeline (subsampling, semi-parametric bootstrap, and computation and aggregation of the confidence intervals) is hosted in the code repository. This record provides the resulting static data together with two lightweight R scripts, so that the reported figures can be reproduced directly, without rerunning the computationally expensive pipeline. Contents (one folder per use case) Each use-case folder (cifar10, simpsons, superconductor) contains: the four individually estimated confidence intervals per subsample: t_mean_CI_df (Student-t interval for the mean), exact_np_CI_df (non-parametric exact interval for quantiles), np_asympt_norm_CI_df (non-parametric asymptotic interval for quantiles), and bootstrap_single_CI (semi-parametric bootstrap interval); overview_signTrafo.Rdata: the aggregated results (empirical coverage and normalized interval length) that directly produce the paper figures; true_statistics.csv: ground-truth values per experiment (wide format), used to normalize interval lengths and to filter experiments; the empirical performance distribution of the experiments (<case>.json / <case>.Rdata). Reproduction (two stages) Stage 1 — figures: figures/plot-uses-cases.R reads the three overview_signTrafo.Rdata files and reproduces the two main figures (empirical coverage and normalized average interval length). Stage 2 — aggregation: aggregate_single_CIs/aggregate-single-CIs-to-overview.R reconstructs overview_signTrafo.Rdata from the four individual-CI files, so that the figures can be verified to follow from the individual intervals. Both scripts are run from the record root with R (>= 4.3); no GPU and no large-scale computation are required. Note The raw semi-parametric bootstrap draws underlying the intervals (on the order of one hundred gigabytes across all use cases) are not required to reproduce the reported results and are available from the authors on request. Layout BeyondPoint-data/ ├── figures/ │ └── plot-uses-cases.R ├── aggregate_single_CIs/ │ └── aggregate-single-CIs-to-overview.R ├── cifar10/ (classification, CIFAR-10) ├── simpsons/ (classification, Simpsons) └── superconductor/ (regression, Superconductor)