orpo-text-pairs

# ORPO Text Preference Pairs This dataset contains **preference pairs** for training language models using ORPO (Odds Ratio Preference Optimization), DPO, or similar preference-based alignment methods. ## Dataset Description - **Rows**: 8,249 preference pairs (filtered/refined) - **Format**: JSONL - **Language**: English - **Task**: Text-only preference learning (no images) ## Schema Each row contains: | Field | Type | Description | |-------|------|-------------| | `prompt` | list[dict] | Chat messages (user turn) | | `chosen` | list[dict] | Preferred response | | `rejected` | list[dict] | Non-preferred response | | `meta` | dict | Metadata including source dataset, models used, judge info | ### Meta Fields - `dataset`: Source dataset name - `row_index`: Original row index in source - `has_image`: Always `false` for this dataset - `chosen_model` / `rejected_model`: Models that generated responses - `judge_choice_1st` / `judge_choice_2nd`: Judge decisions - `trainable`: Whether the sample is suitable for training ## Source Datasets This dataset is derived from the following source datasets. Please respect each dataset's license terms: | Dataset | License | Link | |---------|---------|------| | **HelpSteer2** | CC-BY-4.0 | [nvidia/HelpSteer2](https://huggingface.co/datasets/nvidia/HelpSteer2) | | **MathInstruct** | MIT | [TIGER-Lab/MathInstruct](https://huggingface.co/datasets/TIGER-Lab/MathInstruct) | | **CodeIO-PyEdu-Reasoning** | ODC-BY | [hkust-nlp/CodeIO-PyEdu-Reasoning](https://huggingface.co/datasets/hkust-nlp/CodeIO-PyEdu-Reasoning) | | **MathV360K** | See original | [Zhiqiang007/MathV360K](https://huggingface.co/datasets/Zhiqiang007/MathV360K) | ## License This dataset is released under mixed licenses inherited from the source datasets: - **HelpSteer2**: CC-BY-4.0 (attribution required) - **CodeIO-PyEdu-Reasoning**: ODC-BY (attribution required) - **MathInstruct**: MIT - **MathV360K**: Please check the original dataset **Attribution is required** when using samples derived from HelpSteer2 or CodeIO-PyEdu-Reasoning. ## Usage ```python from datasets import load_dataset dataset = load_dataset("mncai/orpo-text-pairs") ``` ## Citation If you use this dataset, please cite the original source datasets: ```bibtex @article{wang2024helpsteer2, title={HelpSteer2: Open-source dataset for training top-performing reward models}, author={Wang, Zhilin and Dong, Yi and Zeng, Olivier and Delalleau, Jiaqi and Dolber, Gerald and Shekhar, Malay and Narayanan, Radha and others}, journal={arXiv preprint arXiv:2406.08673}, year={2024} } @article{yue2023mammoth, title={MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuning}, author={Xiang Yue, Xingwei Qu, Ge Zhang, Yao Fu, Wenhao Huang, Huan Sun, Yu Su, Wenhu Chen}, journal={arXiv preprint arXiv:2309.05653}, year={2023} } @article{li2025codeio, title={CodeI/O: Condensing Reasoning Patterns via Code Input-Output Prediction}, author={Li, Junlong and Guo, Daya and Yang, Dejian and Xu, Runxin and Wu, Yu and He, Junxian}, journal={arXiv preprint arXiv:2502.07316}, year={2025} } ```