OpenBB/OpenBB-215K

--- language: - zh - en license: apache-2.0 task_categories: - text-generation - question-answering size_categories: - 100K<n<1M tags: - llm-training - pretrain - sft - dpo - reasoning - chain-of-thought - bilingual pretty_name: OpenBB-215K --- # OpenBB-215K v0.1 **A Bilingual Training Dataset for End-to-End LLM Training Pipeline Research** > Curated by [OpenBB](https://huggingface.co/OpenBB) • Seed: 42 • License: Apache 2.0 --- ## Overview **OpenBB-215K** is a curated, reproducible training dataset designed to drive the complete four-phase LLM training pipeline — pretrain, SFT, alignment, and reasoning extension — in a single, self-contained package. It provides balanced Chinese–English coverage across ~215,000 samples drawn from 9 publicly available sources. The dataset is intentionally compact (~1.2 GB) to enable full-pipeline experiments on a single GPU within hours, while preserving the diversity and quality characteristics of production-scale training data. --- ## Dataset Composition | Phase | Source | Samples | Language | |-------|--------|---------|----------| | **Pretrain** | FineWeb-Edu-Chinese v2.1 | 50,000 | zh | | | FineWeb-Edu sample-10BT | 50,000 | en | | **SFT** | SmolTalk-Chinese | 10,978 | zh | | | UltraChat-200K | 39,022 | en | | **Align** | UltraFeedback Binarized | 50,000 | en | | **Extend** | Opus 4.6 Reasoning (filtered) | 2,326 | en | | | STEM-Reasoning-Complex | 5,000 | en+zh | | | Gemini 3 Pro Reasoning | 5,000 | en | | | Gemini 3.1 Pro Reasoning | 3,120 | en | | **Total** | **9 sources** | **~215,446** | **zh + en** | --- ## File Structure ``` openbb-215k/ ├── README.md ├── pretrain.jsonl # 100K lines (50K zh + 50K en, shuffled) ├── sft.jsonl # 50K lines (11K zh + 39K en, shuffled) ├── align.jsonl # 50K lines (chosen/rejected preference pairs) └── extend.jsonl # 15.4K lines (Chain-of-Thought reasoning) ``` --- ## Data Format ### pretrain.jsonl — Pretraining corpus ```json {"text": "A passage of educational text in Chinese or English..."} ``` ### sft.jsonl — Supervised fine-tuning conversations ```json {"conversations": [{"role": "user", "content": "..."}, {"role": "assistant", "content": "..."}]} ``` ### align.jsonl — Preference pairs for DPO/RLHF ```json {"chosen": [{"role": "user", "content": "..."}, {"role": "assistant", "content": "..."}], "rejected": [{"role": "user", "content": "..."}, {"role": "assistant", "content": "..."}]} ``` ### extend.jsonl — Chain-of-Thought reasoning ```json {"instruction": "Problem statement...", "thinking": "Step-by-step reasoning...", "output": "Final answer..."} ``` | Field | Required | Description | |-------|:--------:|-------------| | `instruction` | ✅ | Problem or task description | | `thinking` | ✅ | Chain-of-Thought reasoning process | | `output` | ✅ | Final answer or solution | --- ## Sources & Attribution All data is sourced from publicly available datasets on HuggingFace Hub. | Source | HuggingFace ID | License | |--------|---------------|---------| | FineWeb-Edu-Chinese v2.1 | [`opencsg/Fineweb-Edu-Chinese-V2.1`](https://huggingface.co/datasets/opencsg/Fineweb-Edu-Chinese-V2.1) | Apache 2.0 | | FineWeb-Edu sample-10BT | [`HuggingFaceFW/fineweb-edu`](https://huggingface.co/datasets/HuggingFaceFW/fineweb-edu) | ODC-By 1.0 | | SmolTalk-Chinese | [`opencsg/Smoltalk-Chinese`](https://huggingface.co/datasets/opencsg/Smoltalk-Chinese) | Apache 2.0 | | UltraChat-200K | [`HuggingFaceH4/ultrachat_200k`](https://huggingface.co/datasets/HuggingFaceH4/ultrachat_200k) | MIT | | UltraFeedback Binarized | [`HuggingFaceH4/ultrafeedback_binarized`](https://huggingface.co/datasets/HuggingFaceH4/ultrafeedback_binarized) | MIT | | Opus 4.6 Reasoning | [`nohurry/Opus-4.6-Reasoning-3000x-filtered`](https://huggingface.co/datasets/nohurry/Opus-4.6-Reasoning-3000x-filtered) | Community | | STEM-Reasoning-Complex | [`galaxyMindAiLabs/stem-reasoning-complex`](https://huggingface.co/datasets/galaxyMindAiLabs/stem-reasoning-complex) | Community | | Gemini 3 Pro Reasoning | [`Roman1111111/gemini-3-pro-10000x-hard-high-reasoning`](https://huggingface.co/datasets/Roman1111111/gemini-3-pro-10000x-hard-high-reasoning) | Community | | Gemini 3.1 Pro Reasoning | [`Roman1111111/gemini-3.1-pro-hard-high-reasoning`](https://huggingface.co/datasets/Roman1111111/gemini-3.1-pro-hard-high-reasoning) | Community | > Extend phase reasoning data is sourced from community-contributed synthetic datasets publicly available on HuggingFace Hub. --- ## Reproducibility The dataset is fully reproducible from public sources: ```bash python openbb/tools/build_openbb_215k.py ``` All sampling uses `random.seed(42)` and streams data directly from HuggingFace Hub. Running the build script on any machine will produce identical output. --- ## Quick Start ```bash # Phase 1: Pretrain bb train tier=nano stage=pretrain data_path=openbb/data/openbb-215k/pretrain.jsonl # Phase 2: Supervised Fine-Tuning bb train tier=nano stage=sft data_path=openbb/data/openbb-215k/sft.jsonl # Phase 3: Preference Alignment (DPO) bb train tier=nano stage=align method=dpo data_path=openbb/data/openbb-215k/align.jsonl # Phase 4: Reasoning Extension (CoT) bb train tier=nano stage=extend method=reason data_path=openbb/data/openbb-215k/extend.jsonl ``` --- ## Citation ```bibtex @misc{openbb215k, title = {OpenBB-215K: A Bilingual Training Dataset for End-to-End LLM Pipeline Research}, author = {OpenBB Team}, year = {2026}, url = {https://huggingface.co/datasets/OpenBB/OpenBB-215K}, } ``` --- ## License This dataset is released under the [Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0). Individual source datasets retain their original licenses as listed in the attribution table above.language: - 中文 - 英文 license: apache-2.0 task_categories: - 文本生成 - 问答 size_categories: - 100K<n<1M tags: - 大语言模型训练 - 预训练 - 监督微调（SFT） - 直接偏好优化（DPO） - 推理 - 思维链（Chain-of-Thought） - 双语 pretty_name: OpenBB-215K # OpenBB-215K v0.1 **面向端到端大语言模型（Large Language Model）训练流水线研究的双语训练数据集** > 由 [OpenBB](https://huggingface.co/OpenBB) 整理 • 随机种子：42 • 许可证：Apache 2.0 --- ## 概览 **OpenBB-215K** 是一款经过精心整理、可复现的训练数据集，旨在以单一自包含的套件支撑完整的四阶段大语言模型训练流水线——预训练、监督微调（SFT）、对齐以及推理扩展。该数据集包含约21.5万个样本，取自9个公开数据源，实现了中文与英文的均衡覆盖。 该数据集刻意控制了体积（约1.2 GB），可在单张GPU上于数小时内完成全流水线实验，同时保留了生产级训练数据的多样性与质量特性。 --- ## 数据集构成 | 阶段 | 数据源 | 样本数 | 语言 | |-------|--------|---------|----------| | **预训练** | FineWeb-Edu-Chinese v2.1 | 50,000 | 中文 | | | FineWeb-Edu sample-10BT | 50,000 | 英文 | | **监督微调（SFT）** | SmolTalk-Chinese | 10,978 | 中文 | | | UltraChat-200K | 39,022 | 英文 | | **对齐** | UltraFeedback Binarized | 50,000 | 英文 | | **扩展** | Opus 4.6 Reasoning (filtered) | 2,326 | 英文 | | | STEM-Reasoning-Complex | 5,000 | 中英双语 | | | Gemini 3 Pro Reasoning | 5,000 | 英文 | | | Gemini 3.1 Pro Reasoning | 3,120 | 英文 | | **总计** | **9个数据源** | **约215,446** | **中英双语** | --- ## 文件结构 openbb-215k/ ├── README.md ├── pretrain.jsonl # 100K行（50K中文 + 50K英文，已随机打乱） ├── sft.jsonl # 50K行（11K中文 + 39K英文，已随机打乱） ├── align.jsonl # 50K行（选中/拒绝偏好对） └── extend.jsonl # 15.4K行（思维链推理数据） --- ## 数据格式 ### pretrain.jsonl — 预训练语料 json {"text": "一段中文或英文的教育文本..."} ### sft.jsonl — 监督微调（SFT）对话数据 json {"conversations": [{"role": "user", "content": "..."}, {"role": "assistant", "content": "..."}]} ### align.jsonl — 用于直接偏好优化（DPO）/基于人类反馈的强化学习（RLHF）的偏好对 json {"chosen": [{"role": "user", "content": "..."}, {"role": "assistant", "content": "..."}], "rejected": [{"role": "user", "content": "..."}, {"role": "assistant", "content": "..."}]} ### extend.jsonl — 思维链（Chain-of-Thought）推理数据 json {"instruction": "Problem statement...", "thinking": "Step-by-step reasoning...", "output": "Final answer..."} | 字段 | 是否必填 | 描述 | |-------|:--------:|-------------| | `instruction` | ✅ | 问题或任务描述 | | `thinking` | ✅ | 思维链推理过程 | | `output` | ✅ | 最终答案或解决方案 | --- ## 数据源与归属声明 所有数据均取自HuggingFace Hub上的公开数据集。 | 数据源 | HuggingFace 数据集标识 | 许可证 | |--------|---------------|---------| | FineWeb-Edu-Chinese v2.1 | [`opencsg/Fineweb-Edu-Chinese-V2.1`](https://huggingface.co/datasets/opencsg/Fineweb-Edu-Chinese-V2.1) | Apache 2.0 | | FineWeb-Edu sample-10BT | [`HuggingFaceFW/fineweb-edu`](https://huggingface.co/datasets/HuggingFaceFW/fineweb-edu) | ODC-By 1.0 | | SmolTalk-Chinese | [`opencsg/Smoltalk-Chinese`](https://huggingface.co/datasets/opencsg/Smoltalk-Chinese) | Apache 2.0 | | UltraChat-200K | [`HuggingFaceH4/ultrachat_200k`](https://huggingface.co/datasets/HuggingFaceH4/ultrachat_200k) | MIT | | UltraFeedback Binarized | [`HuggingFaceH4/ultrafeedback_binarized`](https://huggingface.co/datasets/HuggingFaceH4/ultrafeedback_binarized) | MIT | | Opus 4.6 Reasoning | [`nohurry/Opus-4.6-Reasoning-3000x-filtered`](https://huggingface.co/datasets/nohurry/Opus-4.6-Reasoning-3000x-filtered) | 社区协议 | | STEM-Reasoning-Complex | [`galaxyMindAiLabs/stem-reasoning-complex`](https://huggingface.co/datasets/galaxyMindAiLabs/stem-reasoning-complex) | 社区协议 | | Gemini 3 Pro Reasoning | [`Roman1111111/gemini-3-pro-10000x-hard-high-reasoning`](https://huggingface.co/datasets/Roman1111111/gemini-3-pro-10000x-hard-high-reasoning) | 社区协议 | | Gemini 3.1 Pro Reasoning | [`Roman1111111/gemini-3.1-pro-hard-high-reasoning`](https://huggingface.co/datasets/Roman1111111/gemini-3.1-pro-hard-high-reasoning) | 社区协议 | > 扩展阶段的推理数据取自HuggingFace Hub上公开的社区贡献合成数据集。 --- ## 复现方式 该数据集可完全通过公开数据源复现： bash python openbb/tools/build_openbb_215k.py 所有采样均使用`random.seed(42)`，并直接从HuggingFace Hub流式读取数据。在任意设备上运行该构建脚本均可得到完全一致的输出结果。 --- ## 快速上手 bash # 阶段1：预训练 bb train tier=nano stage=pretrain data_path=openbb/data/openbb-215k/pretrain.jsonl # 阶段2：监督微调（SFT） bb train tier=nano stage=sft data_path=openbb/data/openbb-215k/sft.jsonl # 阶段3：偏好对齐（直接偏好优化DPO） bb train tier=nano stage=align method=dpo data_path=openbb/data/openbb-215k/align.jsonl # 阶段4：推理扩展（思维链CoT） bb train tier=nano stage=extend method=reason data_path=openbb/data/openbb-215k/extend.jsonl --- ## 引用格式 bibtex @misc{openbb215k, title = {OpenBB-215K: A Bilingual Training Dataset for End-to-End LLM Pipeline Research}, author = {OpenBB Team}, year = {2026}, url = {https://huggingface.co/datasets/OpenBB/OpenBB-215K}, } --- ## 许可证 本数据集采用 [Apache许可证2.0](https://www.apache.org/licenses/LICENSE-2.0) 发布。各原始数据源数据集仍保留其在上述归属表中注明的原有许可证。