DeepSeek-V4-Pro-Distilled-200K

<div align="center"> # DeepSeek‑V4‑Pro‑Distilled‑200K ### High-quality Math & STEM reasoning distilled from DeepSeek‑V4‑Pro in Max mode [](#overview) [](#dataset-at-a-glance) [](#dataset-structure) [](#data-schema) **Reasoning traces · Proofs · Verification · Mathematics · Physics · Chemistry · Biology** </div> ## Overview `DeepSeek‑V4‑Pro‑Distilled‑200K` is a supervised fine-tuning collection of long-form mathematical and scientific reasoning. Its responses were generated with **DeepSeek‑V4‑Pro in Max inference mode**, then normalized into a compact conversational schema for model distillation. The release is intentionally simple: every example belongs to one of only two configurations—`Math` or `STEM`—and every record uses the same six top-level fields. ## Dataset at a Glance | | `Math` | `STEM` | Total | |---|---:|---:|---:| | Records | 158,383 | 29,644 | **188,027** | | Shards | 32 | 6 | **38** | | Split | train | train | train | | Teacher | DeepSeek‑V4‑Pro‑Max | DeepSeek‑V4‑Pro‑Max | — |  ## Data Schema Every row contains exactly six top-level fields: | Field | Type | Description | |---|---|---| | `id` | string | Stable record identifier | | `conversations` | list | One `human` turn followed by one `gpt` turn | | `input` | string | User prompt | | `output` | string | Teacher-generated reasoning response and final answer | | `domain` | string | Fine-grained source domain | | `meta` | object | Minimal provenance metadata for the selected configuration | ```json { "id": "sample-id", "conversations": [ {"from": "human", "value": "Solve the problem..."}, {"from": "gpt", "value": "<think>...</think>\nFinal answer..."} ], "input": "Solve the problem...", "output": "<think>...</think>\nFinal answer...", "domain": "math", "meta": { "license": "cc-by-4.0", "dataset": "Nemotron-SFT-Math-v4", "teacher_model": "DeepSeek-V4-Pro-Max" } } ``` ## Quality Controls - 188,027 unique IDs and 188,027 unique normalized inputs. - Twenty-four duplicated normalized inputs removed globally. - Conversation turns match `input` and `output` exactly. - Metadata rebuilt from strict configuration-specific allowlists. - Teacher label normalized to `DeepSeek-V4-Pro-Max` on every record. - All gzip JSONL shards passed streaming parse and schema validation. ## Downstream Reference Results The following results measure a **9B reference checkpoint produced by this DeepSeek‑V4‑Pro distillation program**. They are included as practical downstream evidence, not as a controlled dataset ablation: model architecture, training recipe, decoding settings, context length, and quantization can all affect the result. ### GSM8K The reference checkpoint achieved **95.28% mean exact-answer accuracy** over four independent passes of the complete 1,319-question GSM8K test split.  | Run | Correct | Accuracy | |---:|---:|---:| | 1 | 1,256 / 1,319 | 95.22% | | 2 | 1,252 / 1,319 | 94.92% | | 3 | 1,263 / 1,319 | 95.75% | | 4 | 1,256 / 1,319 | 95.22% | | **Mean** | **5,027 / 5,276** | **95.28%** | For a cross-model comparison, the published reference scores below are shown alongside the distilled checkpoint. The **DeepSeek‑V4‑Pro distilled 9B** row uses the same **95.28%** mean accuracy reported in the four-run table above. | Model | Accuracy | |---|---:| | MiMo-V2.5-Pro | 99.60% | | Llama-3.1-405B-Instruct | 96.80% | | Llama-3.3-70B-Instruct | 94.84% | | **DeepSeek‑V4‑Pro distilled 9B** | **95.28%** | | DeepSeek-V4-Pro | 92.60% | | DeepSeek-V3 | 89.30% | Evaluation settings: `temperature=1.0`, `top_p=0.95`, 32K context, Q8 MTP checkpoint, exact normalized numeric-answer matching. ### MMLU‑Pro selected subjects On the 1,500-question comparison covering Math, Physics, and Chemistry, the distilled checkpoint reached **90.53% average accuracy**. Under the reported evaluation, it led Qwen3.5‑9B official by **0.93 percentage points**, Claude Mythos-distilled 27B by **4.33 points**, and Claude Mythos-distilled 9B by **9.46 points**. These results are downstream reference measurements rather than a controlled dataset ablation.  | Model | Math | Physics | Chemistry | Average | |---|---:|---:|---:|---:| | **DeepSeek‑V4‑Pro distilled 9B** | **92.4%** | **89.4%** | **89.8%** | **90.53%** | | Qwen3.5‑9B official | 90.6% | 89.0% | 89.2% | 89.60% | | Claude Mythos-distilled 27B | 88.4% | 87.6% | 82.6% | 86.20% | | Claude Mythos-distilled 9B | 86.0% | 78.6% | 78.6% | 81.07% | Each model answered the same 500 questions per subject, for 1,500 questions per model. The comparison values and display names are synchronized with the [DeepSeek‑V4‑Pro‑Qwen3.5‑9B model card](https://huggingface.co/Jackrong/DeepSeek-V4-Pro-Qwen3.5-9B-MTP-GGUF). The source card reports the 9B rows at 8K context and Claude Mythos-distilled 27B at 32K context, so the comparison should be interpreted under those documented evaluation builds rather than as a strict architecture-only comparison. ## Loading ```python from datasets import load_dataset math = load_dataset( "Jackrong/DeepSeek-V4-Pro-Distilled-200K", "Math", split="train", token=True, ) stem = load_dataset( "Jackrong/DeepSeek-V4-Pro-Distilled-200K", "STEM", split="train", token=True, ) ``` ## Intended Use - Supervised fine-tuning for mathematical and scientific reasoning. - Proof generation, critique, and verification research. - Long-form chain-of-thought and tool-integrated reasoning experiments. - Controlled Math/STEM mixture studies. ## Limitations - Model-generated answers can contain factual, mathematical, scientific, or reasoning errors. - The release is strongly weighted toward Math. - Exact-input deduplication does not remove all semantic near-duplicates. - Long reasoning traces may not be appropriate for concise production assistants. - Benchmark results are reference-checkpoint measurements, not causal proof of improvement from this dataset alone. - Benchmark comparisons should only be interpreted under the documented evaluation and extraction settings. ## Sources, Licensing & Attribution The consolidated records were selected and transformed from the following source datasets. Their original cards remain authoritative for creation methodology, attribution, ownership, and terms: 1. [nvidia/Nemotron‑Math‑Proofs‑v2](https://huggingface.co/datasets/nvidia/Nemotron-Math-Proofs-v2) — CC BY 4.0. 2. [nvidia/Nemotron‑SFT‑Math‑v4](https://huggingface.co/datasets/nvidia/Nemotron-SFT-Math-v4) — a record-level mixture of CC BY 4.0 and CC BY-SA 4.0. 3. [nvidia/Nemotron‑SFT‑Science‑v2](https://huggingface.co/datasets/nvidia/Nemotron-SFT-Science-v2) — CC BY-SA 4.0. The per-record `meta.license` value is authoritative when filtering or redistributing examples. Users are responsible for satisfying all attribution and share-alike obligations attached to the records they use. The source datasets were published by NVIDIA Corporation; this consolidated release does not imply endorsement by NVIDIA or DeepSeek. ## Disclaimer This dataset is provided for research and development. Validate data and model outputs independently before consequential use.