hy3-sft-1500x

# Tencent Hy3 Diverse SFT 1500 Diverse synthetic SFT traces distilled from **tencent/hy3** through OpenRouter. Each row is already formatted as a complete `system` / `user` / `assistant` conversation and can be passed directly to a tokenizer's chat template. The prompt mix is intentionally task-diverse rather than benchmark- or multiple-choice-shaped. Unlike summary-style distillation sets, the reasoning here is the teacher's **native chain-of-thought**: the end-user prompt was sent unmodified and Hy3's own reasoning stream was captured, not a post-hoc explanation. The assistant response uses a consistent reasoning format: ```text <think> deliberate reasoning trace </think> final response ``` ## Dataset summary - **Rows:** 1,500 - **Task families:** 10 - **Language:** English - **Teacher model:** `tencent/hy3` - **Format:** chat-template-ready JSONL - **Generated:** 2026-07-16 - **Generator:** OpenStill - **Source file:** `sft.jsonl` OpenStill is Empero's internal distillation toolkit and is not publicly available yet. This release is a 1,500-row split of a larger run: 4,347 completed traces from a 5,000-prompt generation corpus, sampled with a fixed seed and stratified by task family so the split preserves the full set's task distribution. Only completed, structurally valid SFT rows are included. ## Using it for SFT The `messages` field is ready for `apply_chat_template`; no DPO conversion or prompt reconstruction is required. ```python from datasets import load_dataset from transformers import AutoTokenizer dataset = load_dataset("json", data_files="sft.jsonl", split="train") tokenizer = AutoTokenizer.from_pretrained("YOUR_BASE_MODEL") def render(example): return { "text": tokenizer.apply_chat_template( example["messages"], tokenize=False, add_generation_prompt=False, ) } dataset = dataset.map(render) ``` For trainers that accept conversational datasets directly, use `messages` without mapping it to a text field. ## Fields | field | description | |---|---| | `id` | Stable trace identifier from the source prompt corpus | | `topic` | Task family and generated domain, separated by ` / ` | | `model` | Teacher model used to generate the trace (`tencent/hy3`) | | `messages` | Ordered chat turns with `system`, `user`, and `assistant` roles | | `messages[0].content` | Generic system instruction (`You are a helpful assistant.`) | | `messages[1].content` | Diverse end-user request | | `messages[2].content` | Full `<think>...</think>` reasoning trace followed by the final response | ## Task distribution | task family | rows | what it contains | |---|---:|---| | `coding_and_debugging` | 225 | Implementation, debugging, refactoring, and code review | | `quantitative_problem_solving` | 225 | Applied mathematics, statistics, estimation, and numerical reasoning | | `planning_and_decision_making` | 180 | Practical plans with constraints, resources, and tradeoffs | | `scientific_and_technical_reasoning` | 180 | Science and engineering explanations, calculations, and predictions | | `analysis_and_argument` | 150 | Evidence-based analysis of claims, scenarios, and competing explanations | | `extraction_and_structured_output` | 150 | JSON, tables, classification, extraction, and data transformation | | `writing_and_transformation` | 121 | Rewriting, editing, summarization, and audience-aware drafting | | `troubleshooting_and_diagnosis` | 119 | Diagnosis from symptoms, logs, or operational context | | `logic_and_constraints` | 75 | Scheduling, ordering, logic, and constraint-satisfaction tasks | | `creative_generation_with_constraints` | 75 | Creative artifacts with testable structural or stylistic requirements | ## Statistics | measurement | minimum | median | mean | maximum | |---|---:|---:|---:|---:| | Reasoning length (words) | 90 | 182 | 183.5 | 637 | | User prompt length (characters) | 113 | 338 | 356.3 | 900 | | Final response length (characters) | 3 | 479 | 539.0 | 2,584 | All 1,500 rows have the same role sequence: ```text system → user → assistant ``` ## Validation & limitations - Every retained row passed structural validation: valid JSONL, the expected three-message role sequence, a non-empty reasoning trace, a non-empty final response, canonical `<think>` boundaries, no truncation at the generation token limit, and no hidden teacher/model metadata in the reasoning. - Prompt generation rejects quiz wording and A/B/C/D multiple-choice formatting. The dataset is designed for general SFT rather than MMLU-style continuation training. - Because the reasoning is native chain-of-thought, some quantitative rows carry the entire derivation in the reasoning trace and end with a terse final answer (for example, a bare number with units). This is intentional but worth knowing when mixing with datasets that expect verbose final responses. - Structural validation is **not** deterministic correctness verification. Code was not executed, calculations were not checked against reference answers, and open-ended outputs were not judged by a separate model. - The data is synthetic and may contain factual mistakes, imperfect code, unsupported assumptions, or stylistic artifacts from the teacher model. Review or filter it for high-stakes use cases. - The dataset contains explicit reasoning traces. Confirm that the target model, tokenizer, and training recipe handle `<think>` tags as intended. ## Support / Donate If this dataset or tooling helped you, consider supporting the project: - **BTC:** `bc1qx6zepu6sfkvshgdmc4ewu6pk6rpadvpgffpp7v` - **LTC:** `ltc1qv2mefzps2vtjcpwfx8xxdrpplrcvltswm68r7x` - **XMR:** `42Dbm5xg5Nq26fdyzfEU7KBnAJfhi7Cvz5J2ex5CzHXkfKuNEJzYCcmJ1GTbgjFZ5MBx72sdG1G9239Cd6rsZfv4QeDkYJY` ## Links - Project: [https://empero.org](https://empero.org) - Teacher route: `tencent/hy3` - Generated with OpenStill, Empero's internal distillation toolkit (not publicly available yet), via the OpenRouter chat-completions API