swe-bench-verified-mini

SWEBench-verified-mini is a subset of SWEBench-verified that uses 50 instead of 500 datapoints, requires 5GB instead of 130GB of storage and has approximately the same distribution of performance, test pass rates and difficulty as the original dataset.

You can find more details here: [https://github.com/mariushobbhahn/make_swe_bench_verified_mini](https://github.com/mariushobbhahn/make_swe_bench_verified_mini)

If you use the [Inspect implementation](https://github.com/UKGovernmentBEIS/inspect_evals/tree/main/src/inspect_evals/swe_bench), you can merely switch the `dataset: str = "princeton-nlp/SWE-bench_Verified",` to `dataset: str = "MariusHobbhahn/swe-bench-verified-mini",` in the "swe_bench.py" file.

SWEBench-verified-mini 是 SWEBench-verified 的一个子集，它仅包含50条数据样本（而非原版的500条），所需存储空间为5GB（而非原版的130GB），且在性能表现、测试通过率与任务难度的分布上与原版数据集基本保持一致。

你可通过以下链接获取更多详细信息：[https://github.com/mariushobbhahn/make_swe_bench_verified_mini](https://github.com/mariushobbhahn/make_swe_bench_verified_mini)

若你使用 [Inspect 实现](https://github.com/UKGovernmentBEIS/inspect_evals/tree/main/src/inspect_evals/swe_bench)，仅需在 `swe_bench.py` 文件中将 `dataset: str = "princeton-nlp/SWE-bench_Verified",` 修改为 `dataset: str = "MariusHobbhahn/swe-bench-verified-mini",` 即可。