gitgood/git_good_bench-lite

GitGoodBench Lite是一个包含120个样本的数据集，用于评估AI代理在解决git任务中的性能（见支持的场景）。数据集中的样本均匀分布在编程语言Python、Java和Kotlin以及样本类型合并冲突解决和文件提交链上。每个样本类型和编程语言都有20个样本。数据集中的所有数据都是从100个独特的、具有宽松许可的开源GitHub仓库中收集的，这些仓库拥有>=1000个star、>=5个分支、>=10个贡献者，并且不是分叉或存档的仓库。数据集包含两种类型的样本：合并场景和文件提交链场景。合并场景包含一个或多个合并冲突，文件提交链场景包含连续的提交。

GitGoodBench Lite is a dataset of 120 samples designed for evaluating the performance of AI agents in resolving git tasks (see Supported Scenarios). The samples in the dataset are evenly split across the programming languages Python, Java, and Kotlin, as well as the sample types merge conflict resolution and file-commit chain. Each sample type and programming language has 20 samples. All data in this dataset are collected from 100 unique, open-source GitHub repositories with permissive licenses that have >=1000 stars, >=5 branches, >=10 contributors, and are not forks or archived. The dataset contains two types of samples: merge scenarios and file-commit chain scenarios. Merge scenarios contain one or more merge conflicts, and file-commit chain scenarios consist of consecutive commits.