DLFaultBench: A Reproducible Benchmark of Real-World Deep-Learning Bugs

DLFaultBench is a reproducibility benchmark of 640 real-world deep-learning bugs mined from the issue trackers of 87 popular ML/DL repositories across 63 organizations (transformers, JAX, vLLM, DeepSpeed, diffusers, PyTorch Lightning, TorchRL, PyG, detectron2, NumPyro, SDV, and more). Every bug is packaged as a self-contained, runnable reproduction: a minimal script that triggers the bug, the exact dependencies to install, a script that recreates the buggy codebase at the offending commit, and a reproduction trajectory documenting how the bug was reproduced and what was observed. Of the 640 bugs, 533 are verified reproducible on the reference machine; the remaining 107 ship with a documented, categorized reason they could not be reproduced (hardware access, pinned builds, or upstream fixes). Codebases and Python environments are regenerated on demand (clone + checkout + venv) rather than shipped, which is why the whole package is ~48 MB. See index.csv for the canonical bug_id → report/commit/status map. Prepared as an MSR 2027 Mining Challenge dataset.