private_swe_bench_200

# Private SWE Bench 200 A curated dataset of 200 software engineering tasks (100 bug fixes + 100 feature additions) for evaluating LLM-based code generation and repair capabilities. ## Dataset Structure ``` ├── tasks/ # 200 task instances │ └── <instance_id>/ │ ├── task.toml # Task configuration and metadata │ ├── meta.json # Detailed task metadata │ └── instruction.md # Task instruction/prompt ├── images/ # Docker images for task environments │ └── <instance_id>/ │ └── *.tar.gz # Docker image archive └── README.md ``` ## Task Categories - **bug_fix**: 100 instances — debugging tasks requiring identification and repair of software defects - **feature**: 100 instances — feature addition tasks requiring implementation of new functionality ## Languages Tasks span multiple programming languages including Go, Java, JavaScript, Python, Ruby, PHP, and more. ## Usage Each task instance contains: - `task.toml`: Configuration with benchmark metadata, repository info, expected exit codes, and reward values - `meta.json`: Detailed metadata including mutation strategies, difficulty scores, modified files, and test information - `instruction.md`: The natural language instruction describing the task to be performed ## License MIT