Data for Re-evaluating the CaloChallenge 2022 with Statistical Hypothesis Testing

This repository provides the numerical results of a statistical re-evaluation of generative calorimeter shower models submitted to the Fast Calorimeter Simulation Challenge 2022 (CaloChallenge 2022). The results support the study Re-evaluating the CaloChallenge 2022 with Statistical Hypothesis Testing, which investigates the agreement between generated calorimeter showers and Geant4 reference simulations using nonparametric two-sample tests. The archive covers all four evaluation categories: Dataset 1 photons, Dataset 1 pions, Dataset 2 electrons, and Dataset 3 electrons. It contains the results of four statistical metrics: the mean marginal Kolmogorov–Smirnov statistic (KS), sliced Kolmogorov–Smirnov statistic (SKS), sliced Wasserstein distance (SWD), and maximum mean discrepancy (MMD). The results are organized under the results/ directory by dataset, hypothesis, model submission, shower representation, and evaluation sample size. For each dataset, the Null/ directory contains reference-versus-reference evaluations used to characterize the null distributions of the test statistics. The alternative_hypotheses/ directory contains reference-versus-generated evaluations for individual model submissions. Dataset 1 pions additionally includes train_null/ and alternative_hypotheses_train/ directories for evaluations involving the training partition. The archive includes both full-shower comparisons, performed on voxel-level energy deposits, and high-level-feature comparisons involving deposited energy, longitudinal energy profiles, shower centers, shower widths, and sparsity. Results are stored for evaluation batch sizes of 100, 500, 1,000, 5,000, and 10,000 events per sample, with additional 20,000-event configurations in parts of the archive.