MergeCoT数据集

# MergeCoT Dataset A large-scale dataset for training models to resolve git merge conflicts using chain-of-thought (CoT) reasoning. This dataset contains **87,690 examples** across multiple programming languages, with detailed reasoning traces for merge conflict resolution. ## Dataset Summary MergeCoT provides paired examples of: - **Base versions** and two conflicting changes (`a` and `b`) - **Merged results** that correctly combine both changes - **Chain-of-thought reasoning** explaining the merge decision process - Support for **4 programming languages**: Java, JavaScript, C#, TypeScript ## Dataset Statistics | Split | Examples | Size | |-------|----------|------| | Train | 70,150 | 75.3 MB | | Validation | 8,768 | 9.5 MB | | Test | 8,772 | 9.5 MB | | **Total** | **87,690** | **94.3 MB** | ### Language Distribution - **Java**: 31,851 examples (36.3%) - **JavaScript**: 21,520 examples (24.5%) - **C#**: 9,498 examples (10.8%) - **TypeScript**: 7,281 examples (8.3%) ## Dataset Structure ### Data Fields - `a` (string): First modified version of the code - `b` (string): Second modified version of the code - `base` (string): Original base version before modifications - `merged` (string): Correctly merged result combining changes from `a` and `b` - `lang` (string): Programming language (`java`, `javascript`, `csharp`, `typescript`) - `da` (string): Description of changes in version `a` (tokenized diff format) - `db` (string): Description of changes in version `b` (tokenized diff format) - `qwen3_cot` (string): Chain-of-thought reasoning explaining the merge process - `cot_answer` (string): Final answer after reasoning (same as `merged`) - `total_len` (int64): Total token length of the example ### Example ```python { "lang": "javascript", "base": "function calculate(x, y) {\n return x + y;\n}", "a": "function calculate(x, y) {\n // Add numbers\n return x + y;\n}", "b": "function sum(x, y) {\n return x + y;\n}", "merged": "function sum(x, y) {\n // Add numbers\n return x + y;\n}", "qwen3_cot": "1. Analyze conflict: Change A adds a comment, change B renames the function...", "cot_answer": "function sum(x, y) {\n // Add numbers\n return x + y;\n}" } ``` ## Usage ### Load with Hugging Face Datasets ```python from datasets import load_dataset # Load the entire dataset dataset = load_dataset("chunyoupeng/merged-cot") # Access individual splits train_data = dataset["train"] val_data = dataset["val"] test_data = dataset["test"] # Example: Print first training example print(train_data[0]) ``` ### Fine-tuning Example ```python from datasets import load_dataset from transformers import AutoTokenizer, AutoModelForCausalLM # Load dataset dataset = load_dataset("chunyoupeng/merged-cot") # Prepare training data def format_example(example): prompt = f"""Resolve the following git merge conflict: Base version: {example['base']} Change A: {example['a']} Change B: {example['b']} Provide step-by-step reasoning and the merged result:""" response = f"{example['qwen3_cot']}\n\nMerged result:\n{example['merged']}" return {"prompt": prompt, "response": response} formatted_dataset = dataset.map(format_example) # Continue with your training pipeline... ``` ## Dataset Creation This dataset was created using the **MergeCoT** framework, which: 1. Collects real-world git merge scenarios from open-source repositories 2. Generates chain-of-thought reasoning using advanced LLMs 3. Validates merge correctness through automated testing 4. Filters and curates high-quality examples ### Source Data - Real git merge conflicts from popular open-source projects - Covers common scenarios: refactoring, feature additions, bug fixes - Includes both simple and complex merge cases ### Curation Process - Automated extraction of merge conflicts from git history - LLM-based generation of reasoning traces - Quality filtering based on: - Code compilability - Logical consistency - Reasoning quality ## Applications This dataset can be used for: - **Training merge conflict resolution models** - **Teaching LLMs to reason about code** - **Improving developer tools for git workflows** - **Research on chain-of-thought reasoning for code** - **Benchmarking code understanding capabilities** ## Limitations - Limited to 4 programming languages (Java, JavaScript, C#, TypeScript) - May not cover all types of merge conflicts - Reasoning quality depends on the LLM used for generation - Some examples may have simplified conflict scenarios ## Citation If you use this dataset in your research, please cite: ```bibtex @dataset{mergecot2024, title={MergeCoT: A Large-Scale Dataset for Git Merge Conflict Resolution with Chain-of-Thought Reasoning}, author={Chunyou Peng}, year={2024}, publisher={Hugging Face}, url={https://huggingface.co/datasets/chunyoupeng/merged-cot} } ``` ## License This dataset is released under the Apache 2.0 License. ## Contact For questions or feedback, please open an issue on the dataset repository or contact the author. --- **Generated with**: MergeCoT Framework **Last Updated**: November 2024 **Dataset Version**: 1.0