qwen3.8-max-distillation-50k

# Qwen3.8-Max Distillation 50K A curated dataset of **49,772 teacher-generated traces** from `qwen3.8-max-preview`, prepared for supervised fine-tuning and off-policy knowledge distillation. The teacher responses are preserved as returned by the API. Where the model emitted visible `<think>...</think>` blocks, those blocks remain in the assistant message. Some simpler prompts received direct answers without a thinking block. > [!CAUTION] > **Terms and provenance notice — not cleared for unrestricted commercial/model-training use.** This dataset was generated through Alibaba Cloud Model Studio. The current Alibaba Cloud International Product Terms, §4.48(d)(v), restrict using Model Studio, its hosted models, or their outputs to train or develop products or services that compete with Alibaba Cloud or its affiliates unless expressly authorized. In addition, at least 12,106 rows (24.3%) originate from explicitly noncommercial sources: Evol-Instruct-Code (CC BY-NC-SA 4.0) and SciQ (CC BY-NC 3.0). Other sources have heterogeneous attribution and share-alike obligations. Public availability of this repository is **not** a grant of rights. Before training, redistribution, or commercial use, review the agreement applicable to your account/region/plan and every upstream source license. See [`PROVENANCE.md`](PROVENANCE.md) and the [official Model Studio terms](https://www.alibabacloud.com/help/en/legal/latest/alibaba-cloud-international-website-product-terms-of-service-v-3-8-0#d48a5c007bamp). ## Dataset summary | Split | Rows | |---|---:| | Train | 44,796 | | Validation | 2,488 | | Test | 2,488 | | **Total** | **49,772** | ### Domain distribution | Domain | Rows | Share | |---|---:|---:| | Math | 16,597 | 33.3% | | Code | 14,057 | 28.2% | | Reasoning | 11,690 | 23.5% | | Instruction following | 7,302 | 14.7% | | Tool use | 126 | 0.3% | ### Source distribution The prompt mixture includes: - Math: MetaMathQA, NuminaMath-CoT, OrcaMath, GSM8K, and all seven MATH subject configurations - Code: Evol-Instruct-Code, CodeAlpaca, MBPP, and HumanEval - Reasoning: SciQ, CommonsenseQA, QASC, ARC-Challenge, ARC-Easy, and OpenBookQA - Instruction following: Tulu 3, Dolly, and IFEval - Tool use: synthetic multi-tool scenarios The largest individual sources are Evol-Code (8,138), MetaMathQA (6,406), CodeAlpaca (5,604), Tulu 3 (5,397), and NuminaMath-CoT (4,247). ## Generation - **Teacher:** `qwen3.8-max-preview` - **API:** Alibaba Cloud OpenAI-compatible endpoint - **Temperature:** 0.7 - **Top-p:** 0.95 - **Maximum completion:** 8,192 tokens - **Prompting:** domain-specific system prompts for structured reasoning, final answers, code, and tool calls - **Reasoning budget:** a provider-supported thinking budget of 2,048 tokens was used for most of the generation run; early traces were generated before that cap was introduced - **Raw output policy:** teacher outputs were not rewritten or normalized before curation ## Curation Raw generation produced 53,693 records. The curation pipeline: 1. Required system/user/assistant message structure. 2. Removed incomplete generations (`finish_reason != stop`). 3. Removed empty, very short, and low-substance responses. 4. Applied domain-specific checks, including boxed-answer checks for math and code-presence checks for code tasks. 5. Removed tool-use refusals that did not attempt the task. 6. Assigned a heuristic quality score. 7. Removed exact response duplicates and duplicate prompt prefixes. 8. Randomly created 90%/5%/5% train, validation, and test splits. ### Curated quality indicators | Indicator | Value | |---|---:| | Mean heuristic quality score | 8.59 / 10 | | Score range | 4.5–10.0 | | Responses containing `<think>` | 82.6% | | Math responses containing `\\boxed{...}` | 99.4% | | Code responses containing fenced code | 99.4% | | Average completion length | 957 tokens | The quality score is a **heuristic**, not a factual-correctness judgment. ## Record schema Each row contains: ```python { "id": "stable 16-character trace identifier", "domain": "math | code | reasoning | instruction | tool_use", "source": "source dataset name", "difficulty": "easy | medium | hard | unknown", "messages": [ {"role": "system", "content": "..."}, {"role": "user", "content": "..."}, {"role": "assistant", "content": "..."}, ], "ground_truth": "reference answer when available", "quality_score": 8.5, "teacher_model": "qwen3.8-max-preview", "completion_tokens": 957, "generation_time": "ISO-8601 timestamp", } ``` `ground_truth` is empty for open-ended sources and for some source datasets that did not expose a directly extractable answer. Approximately 19.6% of curated records have no ground-truth string. ## Usage ```python from datasets import load_dataset ds = load_dataset("r0b0tlab/qwen3.8-max-distillation-50k") print(ds) print(ds["train"][0]["messages"]) ``` For response-only SFT, mask the system and user tokens and compute loss only over the assistant response. ## Intended uses - Supervised fine-tuning of chat or reasoning models - Off-policy teacher-to-student distillation - Ablations on reasoning-format and direct-answer mixtures - Research on math, code, scientific reasoning, and instruction following ## Important limitations - **Generated text can be wrong.** The responses were not independently verified problem-by-problem. - `<think>` content is model-emitted visible reasoning prompted by the collection pipeline. It should not be interpreted as privileged access to hidden model state. - The dataset includes prompts originating from standard benchmark splits, including GSM8K, MATH, HumanEval, MBPP, ARC, IFEval, and related sets. A model trained on this dataset must not be fairly evaluated on overlapping benchmark items without contamination controls. - The source mixture is predominantly English but includes some multilingual instruction data. - Tool-use data is only 126 rows after strict curation and is not sufficient by itself for robust function-calling training. - Heuristic filtering favors structured responses and may introduce style bias. - The random train/validation/test split is intended for pipeline development, not uncontaminated capability evaluation. ## Licensing and attribution This is a multi-source derived dataset. The repository does **not** claim to replace or override the licenses, terms, or attribution requirements of the underlying prompt datasets or the teacher-model provider. Users are responsible for reviewing and complying with the terms of every upstream source relevant to their use case. Because the source mixture has heterogeneous upstream terms, the Hub metadata uses `license: other` rather than applying one blanket license to all records. ## Citation ```bibtex @misc{r0b0tlab2026qwen38distillation50k, title = {Qwen3.8-Max Distillation 50K}, author = {r0b0tlab}, year = {2026}, publisher = {Hugging Face}, howpublished = {\url{https://huggingface.co/datasets/r0b0tlab/qwen3.8-max-distillation-50k}} } ```