C2MBench

1. Dataset Summary C2MBENCH is a large-scale benchmark dataset specifically constructed for fine-grained Conventional Commits (CCS) classification research. Based on the CCS specification, it provides high-quality commit data with verified types and scopes across multiple programming languages. The dataset contains 93,807 CCS-compliant commits extracted from 119 high-quality repositories. It spans a wide temporal range (April 2017 to February 2023) and includes 142 distinct commit types and 6,633 unique scopes. 2. Dataset Structure Data Splits We employ a strict temporal split to simulate real-world scenarios, ensuring that repositories are tracked continuously over time. File Count Usage train.json 74,581 Model training valid.json 9,939 Hyperparameter tuning test.json 9,287 Final performance evaluation Language Distribution (by Repository) The benchmark covers 9 key programming languages: Language Repositories Language Repositories TypeScript 61 Rust 2 JavaScript 34 Java 2 Go 11 C++ 1 Python 4 Kotlin 1 C# 3 3. Data Schema Each record in the dataset is a JSON object representing a single commit. Below is a description of the key fields: Field Type Description hash string Unique identifier for the commit. message string The full original commit message. commit_type string The extracted CCS type (e.g., feat, fix, chore). commit_scope string The extracted CCS scope (can be null if not provided). is_CCS integer Binary indicator (1) for CCS compliance. language string Programming language of the repository. repo string Full name of the source repository (Owner/Repo). mods array List of modified files, including diff, new_path, and old_path. date string Timestamp of the commit. license string Original license of the source repository. 4. Construction Principles The construction of C2MBENCH prioritizes Scale, Quality, and Repository Continuity: Keyword Detection: Identifying repositories explicitly adopting CCS ("conventionalcommits.org"). Format Validation: Using a robust parser to validate <type>[optional scope]: <description>. Temporal Alignment: Discarding all commits preceding the repository’s formal adoption of CCS to ensure data purity.