Luth-2-Post-Training-SFT

 # Luth-2-Post-Training-SFT **Luth-2-Post-Training-SFT** is the French supervised fine-tuning mixture used to train [Luth-2-0.8B](https://huggingface.co/kurakurai/Luth-2-0.8B) and [Luth-2-2B](https://huggingface.co/kurakurai/Luth-2-2B). It spans math, code, knowledge, instruction following and tool calling in a single schema, with **1,969,768 examples** and **3.12B training tokens**. - 📄 **Blog**: [Luth-2: Pushing the French Capabilities of SLMs with MOPD](https://huggingface.co/blog/MaxLSB/luth-2) - 🤗 **Models**: [Luth-2-0.8B](https://huggingface.co/kurakurai/Luth-2-0.8B) · [Luth-2-2B](https://huggingface.co/kurakurai/Luth-2-2B) - 📊 **Datasets**: [SFT](https://huggingface.co/datasets/kurakurai/Luth-2-Post-Training-SFT) · [RL](https://huggingface.co/datasets/kurakurai/Luth-2-Post-Training-RL) - 💻 **Code**: [GitHub](https://github.com/kurakurai/Luth-2) - 🏆 **Leaderboard**: [French LLM Leaderboard](https://huggingface.co/spaces/kurakurai/llm_leaderboard_fr) ## Composition | Domain | Examples | % | Tokens | % | | --------------------- | ------------: | ----: | -------------: | ----: | | Knowledge | 595,187 | 30.2% | 870.5M | 27.9% | | Code | 548,432 | 27.8% | 693.0M | 22.2% | | Math | 428,031 | 21.7% | 1,161.1M | 37.2% | | Instruction following | 211,942 | 10.8% | 202.3M | 6.5% | | Tool calling | 186,176 | 9.5% | 196.7M | 6.3% | | **Total** | **1,969,768** | 100% | **3.12B** | 100% | > [!NOTE] > **Decontamination**: 8-gram token overlap against our evaluation benchmark suite. A sample is dropped when more than 50% of any benchmark item's 8-grams appear in it. ## Provenance Prompts were translated to French where the source was English, and assistant turns were **regenerated** with a strong open-source teacher rather than translated. `source` names where the prompt came from, `generator` corresponds to the model that wrote the answer. | Source | Rows | Generator | | --------------------------------------------------- | ------: | ---------------------------- | | `nvidia/Nemotron-Post-Training-Dataset-v2` | 934,387 | `Qwen/Qwen3.6-35B-A3B` | | `allenai/Dolci-Instruct-SFT` | 702,570 | `Qwen/Qwen3-30B-A3B` | | `allenai/Dolci-Instruct-SFT-Tool-Use` | 186,176 | `google/gemma-4-31B-it` | | `HuggingFaceTB/smoltalk2` (French rows) | 50,396 | -- | | `HuggingFaceTB/smoltalk2` (multi-turn IF) | 28,182 | `google/gemma-4-26b-a4b-it` | | `allenai/tulu-3-sft-personas-math` | 47,440 | `Qwen/Qwen3-32B` | | `allenai/tulu-3-sft-personas-instruction-following` | 20,617 | `Qwen/Qwen3-32B` | > [!IMPORTANT] > The dataset is CC-BY-4.0. However, StackOverflow-derived rows from Nemotron remain CC-BY-SA-4.0. ## SFT results Full fine-tuning on top of Qwen3.5-0.8B and Qwen3.5-2B for 1 epoch, `lr=3e-5` cosine, `max_seq_length=16384`, global batch of ~500K tokens, with FlashAttention-3 and sequence packing. Evaluations can be reproduced using our [GitHub](https://github.com/kurakurai/Luth-2) repository. The benchmarks are French subsets or verified translations, scored with `temperature=0.6, top_p=0.95, top_k=20`, thinking disabled, averaged over 10 runs. | | Qwen3.5-0.8B | | Qwen3.5-2B | | | ---------------- | -----------: | ------------: | -----------: | ------------: | | **Stage** | **Baseline** | **After SFT** | **Baseline** | **After SFT** | | MGSM-rev2 | 35.20 | **66.12** | 64.60 | **84.12** | | AIME 24 | 1.00 | **3.00** | 11.67 | **11.67** | | AIME 25 | 0.33 | **2.00** | 6.67 | **12.00** | | Math-500 | 27.46 | **47.04** | 65.06 | **72.70** | | Global-MMLU-Lite | 44.00 | **50.20** | 58.00 | **62.05** | | MMLU-ProX-Lite | 27.60 | **35.63** | 47.20 | **52.70** | | GPQA-Diamond | 23.80 | **27.27** | 36.80 | **39.39** | | IFEval | 44.47 | **59.57** | 61.91 | **64.64** | | Multi-IF | 32.72 | **45.59** | 45.38 | **58.66** | | HumanEval+ | 10.87 | **36.75** | 37.56 | **58.56** | | MBPP+ | 18.20 | **36.85** | 42.12 | **54.58** | | BFCL v2 | 51.49 | **63.20** | 51.78 | **67.73** | ## Contact Questions or feedback? Reach us on LinkedIn: [Maxence Lasbordes](https://www.linkedin.com/in/maxence-lasbordes/) and [Guillaume Pradel](https://www.linkedin.com/in/guillaume-pradel/). ## Citation ```bibtex @misc{luth2, title = {Luth-2: Pushing the French Capabilities of SLMs with MOPD}, author = {Maxence Lasbordes and Guillaume Pradel}, year = {2026}, url = {https://huggingface.co/blog/MaxLSB/luth-2} } ```