SWE-Next

<div align="center"> <h1>SWE-Next: Scalable Real-World Software Engineering Tasks for Agents</h1> </div> <div align="center"> <a href="https://arxiv.org/abs/2603.20691"><img alt="Paper" src="https://img.shields.io/badge/Paper-arXiv-b31b1b?style=for-the-badge&logo=arxiv&logoColor=white"></a> <a href="https://tiger-ai-lab.github.io/SWE-Next/"><img alt="Project Page" src="https://img.shields.io/badge/Project%20Page-Website-4285F4?style=for-the-badge&logo=googlechrome&logoColor=white"></a> <a href="https://github.com/TIGER-AI-Lab/SWE-Next"><img alt="Code" src="https://img.shields.io/badge/Code-GitHub-181717?style=for-the-badge&logo=github&logoColor=white"></a> <a href="https://huggingface.co/datasets/TIGER-Lab/SWE-Next-SFT-Trajectories"><img alt="SFT Trajs" src="https://img.shields.io/badge/SFT%20Trajs-HuggingFace-FFD21E?style=for-the-badge&logo=huggingface&logoColor=000"></a> <a href="https://huggingface.co/TIGER-Lab/SWE-Next-7B"><img alt="Model 7B" src="https://img.shields.io/badge/Model%207B-HuggingFace-FFD21E?style=for-the-badge&logo=huggingface&logoColor=000"></a> <a href="https://huggingface.co/TIGER-Lab/SWE-Next-14B"><img alt="Model 14B" src="https://img.shields.io/badge/Model%2014B-HuggingFace-FFD21E?style=for-the-badge&logo=huggingface&logoColor=000"></a> </div> # SWE-Next Dataset SWE-Next is an execution-grounded dataset of **2,308** self-verifying software engineering tasks mined from real merged GitHub pull requests. Starting from **3,971** seeded Python repositories and **102,582** executed candidate base/merged commit pairs, SWE-Next retains only instances where the merged commit produces a strict test improvement without regressions. The final release spans **311** repositories, with commit dates ranging from 2012 to 2025. Unlike isolated code-generation benchmarks, SWE-Next instances are tied to real repository states and executable environments. Each example packages the task statement, the ground-truth code and test diffs, the relevant test transitions, and the environment artifacts needed for reproducible execution. <div align="center"> <img src="https://raw.githubusercontent.com/TIGER-AI-Lab/SWE-Next/main/docs/static/images/teaser.png" alt="SWE-Next teaser" width="100%" style="max-width: 900px; border-radius: 8px; box-shadow: 0 4px 10px rgba(0,0,0,0.1);"> </div> ## Dataset Overview Each row in `SWE_Next_dataset.jsonl` is a self-verifying SWE task built from a real merged PR. For every retained instance, SWE-Next executes tests on the base and merged commits, identifies the tests that improve, and stores the exact repository-level artifacts required for downstream trajectory collection or agent evaluation. The release is intended for: - training repository-level SWE agents, - collecting agent trajectories under executable environments, - studying execution-grounded task synthesis from real repositories. ## Format and Fields The dataset contains one training split with **2,308** rows. Each row is a JSON object with fields such as: | Field | Type | Description | |---|---|---| | `instance_id` | `string` | Unique identifier in the format `owner__repo-commit_hash` | | `repo` | `string` | GitHub repository in `owner/repo` format | | `base_commit` | `string` | Commit hash of the repository state before the PR was merged | | `commit_hash` | `string` | Commit hash of the merged PR | | `patch` | `string` | Ground-truth code diff introduced by the PR | | `test_patch` | `string` | Ground-truth test diff introduced by the PR | | `problem_statement` | `string` | Natural-language issue description derived from the diff and execution evidence | | `FAIL_TO_PASS` | `list[string]` | Tests that transition from non-passing to passing after the fix | | `PASS_TO_PASS` | `list[string]` | Tests that pass on both base and merged commits | | `dockerfile` | `string` | Self-contained Dockerfile for reproducing the execution environment | | `docker_image` | `string` | Pre-built Docker image for the instance | | `parsed_commit_content` | `string` (JSON) | Structured representation of the commit diff and metadata | | `execution_result_content` | `string` (JSON) | Raw execution results on base and merged commits | | `expected_output_json` | `string` (JSON) | Expected per-test outcomes after applying the patch | | `exec_type` | `string` | Execution label; in this release, retained instances are `NEW_COMMIT_BETTER` | | `created_at` | `string` | ISO 8601 timestamp of the merged commit | ## Files - `SWE_Next_dataset.jsonl`: the released execution-grounded task dataset ## Usage Load the dataset with Hugging Face Datasets: ```python from datasets import load_dataset ds = load_dataset("TIGER-Lab/SWE-Next") print(ds["train"][0].keys()) ``` Each example can be used directly in repository-level agent pipelines that consume task descriptions, patches, test transitions, and Docker-based execution artifacts. ## Relationship to the SWE-Next Release This repo contains the base execution-grounded task dataset. Related artifacts are available separately: - **SFT trajectories**: `TIGER-Lab/SWE-Next-SFT-Trajectories` - **Released models**: `TIGER-Lab/SWE-Next-7B`, `TIGER-Lab/SWE-Next-14B` - **Project code**: `github.com/TIGER-AI-Lab/SWE-Next` ## Citation ```bibtex @misc{liang2026swenextscalablerealworldsoftware, title={SWE-Next: Scalable Real-World Software Engineering Tasks for Agents}, author={Jiarong Liang and Zhiheng Lyu and Zijie Liu and Xiangchao Chen and Ping Nie and Kai Zou and Wenhu Chen}, year={2026}, eprint={2603.20691}, archivePrefix={arXiv}, primaryClass={cs.SE}, url={https://arxiv.org/abs/2603.20691}, } ```