taagarwa/SWE-bench_Verified

SWE-bench Verified是SWE-bench测试集中500个样本的子集，这些样本已经过人工验证以确保质量。SWE-bench是一个用于测试系统自动解决GitHub问题能力的数据集。该数据集收集了来自流行Python仓库的500个测试问题-拉取请求对。评估通过使用拉取请求后的行为作为参考解决方案的单元测试验证来进行。原始SWE-bench数据集是作为SWE-bench的一部分发布的：语言模型能否解决现实世界中的GitHub问题？数据集的结构包括多个特征，如仓库、实例ID、基础提交、补丁、测试补丁、问题陈述、提示文本、创建时间、版本、失败到通过、通过到通过、环境设置提交和难度。数据集支持的任务是在给定完整仓库和GitHub问题的情况下解决问题，文本主要为英文。

SWE-bench Verified is a subset of 500 samples from the SWE-bench test set, which have been human-validated for quality. SWE-bench is a dataset that tests systems’ ability to solve GitHub issues automatically. The dataset collects 500 test Issue-Pull Request pairs from popular Python repositories. Evaluation is performed by unit test verification using post-PR behavior as the reference solution. The original SWE-bench dataset was released as part of SWE-bench: Can Language Models Resolve Real-World GitHub Issues? The dataset structure includes features such as repo, instance_id, base_commit, patch, test_patch, problem_statement, hints_text, created_at, version, FAIL_TO_PASS, PASS_TO_PASS, environment_setup_commit, and difficulty. The dataset supports a task of issue resolution provided a full repository and GitHub issue, and the text is primarily in English.