TMoC/SYNTH-Swallow-Math-Code-Mix

---
license: other
language:
- en
task_categories:
- text-generation
---

# Mixed dataset: SYNTH + SwallowMath-v2 + SwallowCode-v2

This high-signal, all synthetic dataset is a complete shuffled mix of the following four sources:
- **SYNTH** ~63.5%
- **SwallowCode-v2** ~15.5%
- **SwallowMath-v2-textbook** ~10.5%
- **SwallowMath-v2-qa** ~10.0%

The motivation to provide this on HF was the need for a convenient, pre-shuffled merge of the highest quality synthetic / augmented datasets for small language model pre-training experiments as of January 2026. SYNTH by PleIAs will dominate training here, but is supplemented by the excellent Mathematics and Code datasets by tokyotech, which consist of rewritten synthetic samples with consistent and predictable formatting; a very good complement to SYNTH. Taken together like this, the model should acquire general knowledge (SYNTH), fundamental mathematical understanding (Swallow Math V2) and python code skills (Swallow Code V2) that SYNTH in its current form lacks by design.

Details:
- `PleIAs/SYNTH` -> `text = query + synthetic_reasoning + synthetic_answer`
- `tokyotech-llm/swallow-math-v2` -> subsets: `swallow-math-v2-qa`, `swallow-math-v2-textbook`
- `tokyotech-llm/swallow-code-v2` -> subset: `stage5-auto-format`

Single column: `text` to keep training dead simple.

All credit goes to the original authors: [PleIAs](https://huggingface.co/PleIAs) and [tokyotech](https://huggingface.co/tokyotech-llm).