SyncBench
收藏arXiv2025-09-30 收录
下载链接：
https://xhguo7.github.io/SyncMind/
下载链接
链接失效反馈官方服务：
资源简介：
该数据集是一个基准测试，包含了来自21个热门GitHub仓库中的真实世界协作软件工程中的24,332个代理不同步情景实例。此外，该数据集还包括可执行的验证测试以及对大型语言模型（LLM）代理的能力和局限性的洞察。规模上，该数据集拥有24,332个实例，其任务在于评估大型语言模型代理在处理不同步恢复能力方面的表现。
This dataset is a benchmark containing 24,332 instances of agent desynchronization scenarios sourced from real-world collaborative software engineering projects across 21 popular GitHub repositories. Additionally, the dataset includes executable validation tests as well as insights into the capabilities and limitations of large language model (LLM) agents. With a total of 24,332 instances, this benchmark is designed to evaluate the performance of LLM agents in terms of their ability to recover from desynchronization scenarios.
