kimi-mtp-dataset

# Kimi-K2.5 Eagle3 Training Data This dataset contains the instruction-following data used to train an **Eagle3 MTP draft model** for [Kimi-K2.5](https://huggingface.co/moonshotai/Kimi-K2.5) with [TorchSpec](https://github.com/torchspec-project/TorchSpec). All responses were **regenerated by running Kimi-K2.5 via Engine** rather than taken from the original datasets. This is critical for speculative decoding training: the draft model must learn the exact token-level distribution of the target model it is accelerating. The trained Eagle3 draft model is available at [lightseekorg/kimi-k2.5-eagle3](https://huggingface.co/lightseekorg/kimi-k2.5-eagle3). If you find this draft model useful, please give our project **TorchSpec** a 🌟 on [GitHub](https://github.com/torchspec-project/TorchSpec). ## Data source Due to inference resource constraints, some source datasets are only partially regenerated. Here is the list of source datasets used in this mix: | Dataset | Source | # Samples | |---------|--------|-----------| | [mlabonne/open-perfectblend](https://huggingface.co/datasets/mlabonne/open-perfectblend) | `perfectblend` | 296,034 | | [liuhaotian/LLaVA-Instruct-150K](https://huggingface.co/datasets/liuhaotian/LLaVA-Instruct-150K) | `llava_instruct` | 123,102 | | [HuggingFaceTB/smoltalk](https://huggingface.co/datasets/HuggingFaceTB/smoltalk) | `smoltalk_cn` | 48,333 | | [daviddjtzafon/continual-tool-kimi-k2.5](https://huggingface.co/datasets/daviddjtzafon/continual-tool-kimi-k2.5) | `continual_tool_kimi` | 4,370 | | [crownelius/KimiK2.5-2000x-formatted](https://huggingface.co/datasets/crownelius/KimiK2.5-2000x-formatted) | `kimi_2000x` | 2,144 | | [crownelius/Creative-Writing-KimiK2.5-Cleaned](https://huggingface.co/datasets/crownelius/Creative-Writing-KimiK2.5-Cleaned) | `creative_writing` | 1,393 | | [DCAgent2/terminal_bench_2](https://huggingface.co/datasets/DCAgent2/terminal_bench_2__together_ai_moonshotai_Kimi-K2.5_20260203) | `dcagent` | 873 | | [crownelius/Creative-Writing-Reasoning-KimiK2.5-600x](https://huggingface.co/datasets/crownelius/Creative-Writing-Reasoning-KimiK2.5-600x) | `creative_writing_reasoning` | 655 | | **Total** | | **476,904** | ## Data format Each sample contains two fields: - **`conversations`**: a list of turns, each with `from` (`human` / `gpt` / `system`) and `value` (string). - **`source`**: the name of the source dataset (see table above). ```json { "conversations": [ {"from": "human", "value": "What is the capital of France?"}, {"from": "gpt", "value": "The capital of France is Paris."} ], "source": "perfectblend" } ``` Multimodal samples (`llava_instruct`) use OpenAI vision format in the `value` field — a list of `image_url` and `text` objects — with local image paths replaced by public COCO URLs (`http://images.cocodataset.org/train2017/{filename}`). Function-call samples (`continual_tool_kimi`) use Kimi-K2.5's special token format for tool calls: ``` <|tool_calls_section_begin|><|tool_call_begin|>{id}<|tool_call_argument_begin|>{args_json}<|tool_call_end|><|tool_calls_section_end|> ``` Tool results are serialized as `human` turns with the prefix `## Return of {call_id}\n`. ## Training See [TorchSpec](https://github.com/torchspec-project/TorchSpec) for the full training recipe, configuration, and evaluation results. ## License Apache 2.0. All source datasets are Apache 2.0 or MIT licensed.