SWE-Bench_Verified_UTBoost

**Dataset Summary**

In this dataset, we replace some test suites in princeton-nlp/SWE-bench_Verified with augmented test cases to enable a more rigorous evaluation of SWE-Bench. UTBoost was accepted in ACL 2025, and we have opened-sourced the code and metadata in https://github.com/CUHK-Shenzhen-SE/UTBoost.

**Dataset Structure**

An example of a SWE-bench datum is as follows:

```
instance_id: (str) - A formatted instance identifier, usually as repo_owner__repo_name-PR-number.
patch: (str) - The gold patch, the patch generated by the PR (minus test-related code), that resolved the issue.
repo: (str) - The repository owner/name identifier from GitHub.
base_commit: (str) - The commit hash of the repository representing the HEAD of the repository before the solution PR is applied.
hints_text: (str) - Comments made on the issue prior to the creation of the solution PR’s first commit creation date.
created_at: (str) - The creation date of the pull request.
test_patch: (str) - A test-file patch that was contributed by the solution PR.
problem_statement: (str) - The issue title and body.
version: (str) - Installation version to use for running evaluation.
environment_setup_commit: (str) - commit hash to use for environment setup and installation.
FAIL_TO_PASS: (str) - A json list of strings that represent the set of tests resolved by the PR and tied to the issue resolution.
PASS_TO_PASS: (str) - A json list of strings that represent tests that should pass before and after the PR application.
```

**数据集概述**

本数据集针对princeton-nlp/SWE-bench_Verified中的部分测试套件进行替换，采用增强后的测试用例，以实现对SWE-Bench更严谨的评估。本研究提出的UTBoost已被ACL 2025收录，我们已将其代码与元数据开源至https://github.com/CUHK-Shenzhen-SE/UTBoost。

**数据集结构**

一条SWE-bench数据示例如下：

instance_id: (str) - 格式化后的实例标识符，通常采用repo_owner__repo_name-PR-number的格式。
patch: (str) - 修复该问题的标准补丁（即该拉取请求（PR）生成的补丁，不含与测试相关的代码）。
repo: (str) - 来自GitHub的仓库所有者/名称标识符。
base_commit: (str) - 应用修复PR前，仓库HEAD对应的提交哈希值。
hints_text: (str) - 在修复PR的首次提交创建日期之前，针对该议题所发表的评论内容。
created_at: (str) - 该拉取请求（PR）的创建日期。
test_patch: (str) - 由修复PR贡献的测试文件补丁。
problem_statement: (str) - 该议题的标题与正文内容。
version: (str) - 运行评估时所使用的安装版本。
environment_setup_commit: (str) - 用于环境搭建与安装的提交哈希值。
FAIL_TO_PASS: (str) - 一个JSON格式的字符串列表，代表该PR所修复且与该议题解决相关的测试用例集合。
PASS_TO_PASS: (str) - 一个JSON格式的字符串列表，代表在应用PR前后均应通过的测试用例。