ByteDance-Seed/Multi-SWE-bench-flash

该数据集名为Multi-SWE-bench，是一个多语言基准测试集，用于评估大型语言模型（LLMs）在现实世界代码问题解决中的表现。与现有的以Python为中心的基准测试（如SWE-bench）不同，该框架涵盖了7种语言（Java、TypeScript、JavaScript、Go、Rust、C和C++），包含1,632个高质量实例。这些实例是从2,456个候选样本中由68位专家注释者精心挑选的，以确保可靠性。数据集中的每个实例包括组织名称、仓库名称、PR编号、状态、标题、正文、基础分支信息、解决的问题、修复和测试补丁、测试结果等字段。数据集采用CC0许可证，但需遵守所改编开源项目的各自许可证。

The dataset, named Multi-SWE-bench, is a multilingual benchmark designed to evaluate Large Language Models (LLMs) in real-world code issue resolution. Unlike existing Python-centric benchmarks (e.g., SWE-bench), this framework spans 7 languages (Java, TypeScript, JavaScript, Go, Rust, C, and C++) with 1,632 high-quality instances. These instances were curated from 2,456 candidates by 68 expert annotators for reliability. Each data instance includes fields such as organization name, repository name, PR number, state, title, body, base branch information, resolved issues, fix and test patches, test results, and more. The dataset is licensed under CC0, subject to compliance with the respective licenses of the adapted open-source projects.