eth-sri/SWT-bench_bm25_27k_zsb

SWT-bench是一个测试系统自动复现GitHub问题能力的数据集，包含从11个流行的Python GitHub项目中收集的1978个测试Issue-Pull Request对。数据集通过单元测试验证来评估，使用提议的测试模型对测试套件在提交前后的行为进行测试。SWT-bench is a dataset that tests systems’ ability to reproduce GitHub issues automatically, containing 1978 test Issue-Pull Request pairs collected from 11 popular Python GitHub projects. Evaluation is performed by unit test verification using pre- and post-PR behavior of the test suite with and without the proposed tests.