eth-sri/SWT-bench_Verified_bm25_27k_zsb
收藏Hugging Face2025-02-25 更新2025-07-05 收录
下载链接：
https://hf-mirror.com/datasets/eth-sri/SWT-bench_Verified_bm25_27k_zsb
下载链接
链接失效反馈官方服务：
资源简介：
SWT-bench Verified是SWT-bench数据集的一个子集，旨在测试系统自动重现GitHub问题的能力。该数据集收集了来自11个流行的Python GitHub项目的433个测试Issue-Pull Request对。数据集通过单元测试验证来评估，使用测试套件在提出和之后的PR行为中，带有和不带有模型提议的测试。数据集采用了Pyserini的BM25检索进行格式化，并限制了代码上下文大小。
SWT-bench Verified is a subset of the SWT-bench dataset, designed to test systems ability to automatically reproduce GitHub issues. The dataset consists of 433 test Issue-Pull Request pairs from 11 popular Python GitHub projects. Evaluation is performed by unit test verification using the pre- and post-PR behavior of the test suite with and without the proposed tests by the model. The dataset is formatted using Pyserinis BM25 retrieval and has a code context size limit.
提供机构：
eth-sri
