defect4ML

defect4ML是由蒙特利尔理工学院的研究团队创建的一个标准bug基准数据集，包含100个从GitHub和Stack Overflow收集的ML相关bug。该数据集旨在帮助ML系统的实践者和研究者评估他们的测试工具和技术。数据集中的bug涵盖了TensorFlow和Keras框架，涉及多种bug类型和测试属性，如正确性、效率和可重复性。创建过程包括手动检查和过滤，确保每个bug都满足标准基准的严格要求。defect4ML的应用领域包括自动bug修复工具的评估和ML系统测试方法的改进，旨在解决ML系统中的可靠性问题。

Defect4ML is a standard bug benchmark dataset created by a research team at Polytechnique Montréal, which includes 100 ML-related bugs collected from GitHub and Stack Overflow. This dataset is intended to help practitioners and researchers of ML systems evaluate their testing tools and technologies. The bugs in the dataset cover both TensorFlow and Keras frameworks, and involve a variety of bug types and testing attributes such as correctness, efficiency, and reproducibility. The creation process involves manual inspection and filtering to ensure that each bug meets the strict requirements of a standard benchmark. Application areas of Defect4ML include the evaluation of automated bug repair tools and the improvement of ML system testing methods, with the goal of addressing reliability issues in ML systems.