Trigger Anomaly Detection for New Physics at the Large Hadron Collider

For technical details, please see the README.md. This record has a HuggingFace mirror with a dataloader at mirror.In the following, we explain the origin and motivation behind this data set.The Large Hadron Collider (LHC) collides protons 40 million times per second. There exists no storage system that can hold the amount of produced data. Therefore, the CMS experiment at the LHC has a trigger system that filters collisions in real time: fewer than one collision in ten thousand is kept and the rest are permanently discarded. The algorithms that perform this filtering all look for speicifc signatures and contain expectations seeded by theoretical predicitons. A new particle that matches none of the predicted signatures is lost at this first step. Anomaly detection inverts the approach by having a model learn what typical collisions look like and keep whatever departs from them, assuming nothing about what new particles would leave as signatures in the CMS detector. The first such models are already running in the CMS trigger. The issue with all currently produced models pertains to validation. In most anomaly-detection applications, models are validated on recorded examples of anomalies; here a recorded anomaly would itself be a discovery, so no such examples can exist. Simulated anomalies act as a temporary substitute. A model selected on simulations inherits their assumptions, which is the very bias anomaly detection is meant to remove. Validation must therefore rest on typical data alone. This dataset supports that research: it provides a large sample of randomly selected (zero bias) collisions as the trigger records them, a simulation of the same typical data that offers a second, independent view of normality, and twenty simulated anomaly datasets intended for downstream evaluation rather than for model selection or validation.Regardless of this motivation, this data set is rich in statistics, provides a simulation of normal data, and contains a diverse set of simulated anomaly datasets. Therefore, we encourage the use of this data set for general anomaly detection research in machine learning.