GPT-5.5-Gemini-3.1-Pro-Grok-4-Claude-Fable-Mythos-5-Qwen-3.7-Max-Distillation-Cleaned
收藏资源简介：
该数据集是GPT-5.5、Gemini 3.1 Pro、Grok 4、Claude Fable 和 Mythos 5、Qwen 3.7 Max 等模型蒸馏数据的清洗增强版本（v2）。基于原始蒸馏数据集，整合了73个不同的源数据集，经过精确去重、启发式过滤和质量评分，从超过1800万原始样本中缩减至约690万高质量、可直接用于监督微调（SFT）的样本。数据集已预先格式化为聊天格式（messages列），每一条包含用户和助手的对话轮次，可直接用于 Axolotl、Unsloth 或 Hugging Face Trainer 等训练框架，无需额外转换。主要改进包括：采用messages列格式、精确去重减少65%数据（从1980万降至690万）、移除身份映射、单字TRUE/FALSE/垃圾回复、过短输出等低质量样本，并添加了基于启发式的质量评分（0-100）以支持下游过滤。同时保留了来源和原始数据集路径信息。数据包含以下字段：messages（聊天格式对话）、instruction（原始用户指令）、response（原始助手回复）、source（来源拆分名称）、source_dataset（原始Hugging Face数据集路径）、category（内容类别）、quality_score（质量评分）。数据集按类别组织，包括 coding（编程，631万行）、cybersecurity（网络安全，39.6万行）、instruction（通用指令，5.2万行）、distilled（蒸馏响应，6.5万行）等，总计约686万行，774 MB。主要来源包括 vibe_instruct_v2、vibe_coding、high_quality_cybersecurity 等。数据集还提供了训练集（6,481,517样本）和测试集（341,133样本）的划分。该数据集适用于文本生成任务，特别是对话式监督微调，可用于训练编程助手、网络安全助手、通用指令遵循模型等。
This dataset is a cleaned and enhanced version (v2) of distilled data from models such as GPT-5.5, Gemini 3.1 Pro, Grok 4, Claude Fable, Mythos 5, and Qwen 3.7 Max. Based on the original distilled dataset, it integrates 73 different source datasets, undergoing exact deduplication, heuristic filtering, and quality scoring, reducing from over 18 million raw samples to approximately 6.9 million high-quality samples directly usable for supervised fine-tuning (SFT). The dataset is pre-formatted in a chat format (messages column), with each entry containing conversation turns between user and assistant, and can be directly used in training frameworks like Axolotl, Unsloth, or Hugging Face Trainer without additional conversion. Key improvements include: adopting the messages column format, exact deduplication reducing data by 65% (from 19.8 million to 6.9 million), removing identity mappings, single-word TRUE/FALSE/spam replies, overly short outputs, and other low-quality samples, and adding a heuristic-based quality score (0-100) to support downstream filtering. Source and original dataset path information is retained. The data includes the following fields: messages (chat-formatted conversations), instruction (original user instruction), response (original assistant response), source (source split name), source_dataset (original Hugging Face dataset path), category (content category), and quality_score (quality score). The dataset is organized by category, including coding (6.31 million rows), cybersecurity (396,000 rows), instruction (52,000 rows), distilled (65,000 rows), etc., totaling approximately 6.86 million rows and 774 MB. Main sources include vibe_instruct_v2, vibe_coding, high_quality_cybersecurity, etc. The dataset also provides a training set (6,481,517 samples) and a test set (341,133 samples). It is suitable for text generation tasks, especially conversational supervised fine-tuning, and can be used to train programming assistants, cybersecurity assistants, general instruction-following models, and more.
数据集概述
GPT-5.5 Prime-Cleaned Dataset 是一个大规模多模型蒸馏数据集，经过严格清洗，适用于监督微调（SFT）。该数据集以“质量优先于洁净度，洁净度优先于规模”为核心理念，整合了来自多个前沿模型的蒸馏数据。
数据集规模与划分
| 划分 | 样本数 |
|---|---|
| 训练集 (train) | 6,481,517 |
| 测试集 (test) | 341,133 |
原始数据总量为 19,992,128 条，经过去重和质量过滤后，最终保留 6,863,735 条有效数据。
数据字段说明
| 字段 | 类型 | 描述 |
|---|---|---|
| messages | List[Struct{role, content}] | OpenAI 对话格式的多轮对话 |
| source | string | 来源标识符 |
| source_dataset | string | 原始数据集名称 |
| category | string | 内容类别 |
| quality_score | float64 | 质量评分（0-100） |
类别分布与质量评分
| 类别 | 原始数量 | 去重后数量 | 质量过滤后数量 | 平均质量分 |
|---|---|---|---|---|
| coding | 16,853,976 | 6,322,417 | 6,314,576 | 51.9 |
| cybersecurity | 1,869,568 | 401,511 | 396,206 | 55.1 |
| instruction | 454,100 | 62,619 | 52,238 | 53.4 |
| distilled | 179,588 | 65,297 | 65,297 | 59.9 |
| applied | 200,000 | 18,794 | 663 | 52.6 |
| index | 50,000 | 26,109 | 25,349 | 52.5 |
| humanities | 200,000 | 30,653 | 6,248 | 51.1 |
| science | 184,896 | 12,676 | 3,158 | 51.2 |
| 总计 | 19,992,128 | 6,940,076 | 6,863,735 | — |
清洗流程
| 步骤 | 数量 |
|---|---|
| 原始输入 | 19,992,128 |
| 拒绝回答、长度过滤和精确去重移除 | 13,052,052 |
| 初步清洗后 | 6,940,076 |
| 质量过滤移除 | 76,341 |
| 最终数据集 | 6,863,735 |
使用方式
可通过 Hugging Face datasets 库直接加载：
python from datasets import load_dataset
ds = load_dataset("saidutta69/GPT-5.5-Gemini-3.1-Pro-Grok-4-Claude-Fable-Mythos-5-Qwen-3.7-Max-Distillation-Cleaned", split="train")
兼容 Axolotl、LLaMA-Factory、TRL、Unsloth 等支持 OpenAI messages 格式的 SFT 框架。
来源数据集列表
数据集整合了以下 49 个来源数据集：alpaca、autonomy、biology_25k、biology_r2med、chemistry_25k、citation_ground、claude_mythos、claude_opus_48_max_thinking、clydeiii_cybersecurity、conscience_25k、cs_25k、dolly、earth_climate_25k、economics_25k、evolution_25k、fable5_agentic_sft、fable5_repos_full、fenrir_v2_1、gemini35、gemini_pro32、genesis_code、genesis_code_10k、genesis_code_100k、genesis_code_demo、genesis_v11、high_quality_cybersecurity、human_25k、kardashev_25k、kimi_coding、law_25k、legend_python、materials_25k、medical_25k、nano_25k、precinct6_cybersecurity、psychology_25k、renewable_energy_25k、robotics_25k、royal_ghost_501k、science_25k、savani_cyber_attack、species_25k、sports_25k、statistics_25k、supernatural_25k、transport_25k、universe_25k、vibe_coding、vibe_instruct_v2。
许可证
该数据集采用 MIT 许可证。
