UnicornChan/kimi-k2.5-mtp-dataset

---
dataset_info:
features:
- name: conversations
list:
- name: from
dtype: string
- name: value
dtype: string
- name: source
dtype: string
splits:
- name: train
num_examples: 476904
configs:
- config_name: default
data_files:
- split: train
path: data/train-*
license: apache-2.0
language:
- en
- zh
tags:
- speculative-decoding
- eagle3
- kimi-k2.5
- draft-model
- conversations
---

# Kimi-K2.5 Eagle3 Training Data

This dataset contains the instruction-following data used to train an **Eagle3 MTP draft model** for [Kimi-K2.5](https://huggingface.co/moonshotai/Kimi-K2.5) with [TorchSpec](https://github.com/torchspec-project/TorchSpec).

All responses were **regenerated by running Kimi-K2.5 via SGLang** rather than taken from the original datasets. This is critical for speculative decoding training: the draft model must learn the exact token-level distribution of the target model it is accelerating.

The trained Eagle3 draft model is available at [lightseek/kimi-k2.5-eagle](https://huggingface.co/lightseek/kimi-k2.5-eagle).

## Data source

Due to inference resource constraints, some source datasets are only partially regenerated. Here is the list of source datasets used in this mix:

| Dataset | Source | # Samples |
|---------|--------|-----------|
| [mlabonne/open-perfectblend](https://huggingface.co/datasets/mlabonne/open-perfectblend) | `perfectblend` | 296,034 |
| [liuhaotian/LLaVA-Instruct-150K](https://huggingface.co/datasets/liuhaotian/LLaVA-Instruct-150K) | `llava_instruct` | 123,102 |
| [HuggingFaceTB/smoltalk](https://huggingface.co/datasets/HuggingFaceTB/smoltalk) | `smoltalk_cn` | 48,333 |
| [daviddjtzafon/continual-tool-kimi-k2.5](https://huggingface.co/datasets/daviddjtzafon/continual-tool-kimi-k2.5) | `continual_tool_kimi` | 4,370 |
| [crownelius/KimiK2.5-2000x-formatted](https://huggingface.co/datasets/crownelius/KimiK2.5-2000x-formatted) | `kimi_2000x` | 2,144 |
| [crownelius/Creative-Writing-KimiK2.5-Cleaned](https://huggingface.co/datasets/crownelius/Creative-Writing-KimiK2.5-Cleaned) | `creative_writing` | 1,393 |
| [DCAgent2/terminal_bench_2](https://huggingface.co/datasets/DCAgent2/terminal_bench_2__together_ai_moonshotai_Kimi-K2.5_20260203) | `dcagent` | 873 |
| [crownelius/Creative-Writing-Reasoning-KimiK2.5-600x](https://huggingface.co/datasets/crownelius/Creative-Writing-Reasoning-KimiK2.5-600x) | `creative_writing_reasoning` | 655 |
| **Total** | | **476,904** |

## Data format

Each sample contains two fields:

- **`conversations`**: a list of turns, each with `from` (`human` / `gpt` / `system`) and `value` (string).
- **`source`**: the name of the source dataset (see table above).

```json
{
"conversations": [
{"from": "human", "value": "What is the capital of France?"},
{"from": "gpt", "value": "The capital of France is Paris."}
],
"source": "perfectblend"
}
```

Multimodal samples (`llava_instruct`) use OpenAI vision format in the `value` field — a list of `image_url` and `text` objects — with local image paths replaced by public COCO URLs (`http://images.cocodataset.org/train2017/{filename}`).

Function-call samples (`continual_tool_kimi`) use Kimi-K2.5's special token format for tool calls:

```
<|tool_calls_section_begin|><|tool_call_begin|>{id}<|tool_call_argument_begin|>{args_json}<|tool_call_end|><|tool_calls_section_end|>
```

Tool results are serialized as `human` turns with the prefix `## Return of {call_id}\n`.

## Training

See [TorchSpec](https://github.com/torchspec-project/TorchSpec) for the full training recipe, configuration, and evaluation results.

## License

Apache 2.0. All source datasets are Apache 2.0 or MIT licensed.

dataset_info:
特征：
- 名称：conversations，列表类型，包含两个子字段：
- from：数据类型为字符串，用于标识对话角色
- value：数据类型为字符串，用于存储对话内容
- 名称：source：数据类型为字符串，用于标识数据源
数据集划分：
- 划分名称：train，样本数量：476904
配置项：
- 配置名称：default，数据文件：
- 对应划分：train，文件路径：data/train-*
许可证：Apache 2.0
语言：
- 英语
- 中文
标签：
- 推测式解码（speculative-decoding）
- eagle3
- kimi-k2.5
- 草稿模型（draft-model）
- 对话数据

# Kimi-K2.5 Eagle3 训练数据集

本数据集包含用于结合[TorchSpec](https://github.com/torchspec-project/TorchSpec)为[Kimi-K2.5](https://huggingface.co/moonshotai/Kimi-K2.5)训练**Eagle3 MTP草稿模型**的指令遵循数据。

所有回复均通过SGLang调用Kimi-K2.5重新生成，而非直接取自原始数据集。这对推测式解码训练至关重要：草稿模型需要学习其加速的目标模型在Token（Token）级别上的精确分布。

已训练完成的Eagle3草稿模型可在[lightseek/kimi-k2.5-eagle](https://huggingface.co/lightseek/kimi-k2.5-eagle)获取。

## 数据源

受限于推理资源，部分源数据集仅完成了部分重新生成。以下为本数据集混合使用的源数据集列表：

| 数据集 | 数据源标识 | 样本数量 |
|---------|--------|-----------|
| [mlabonne/open-perfectblend](https://huggingface.co/datasets/mlabonne/open-perfectblend) | `perfectblend` | 296,034 |
| [liuhaotian/LLaVA-Instruct-150K](https://huggingface.co/datasets/liuhaotian/LLaVA-Instruct-150K) | `llava_instruct` | 123,102 |
| [HuggingFaceTB/smoltalk](https://huggingface.co/datasets/HuggingFaceTB/smoltalk) | `smoltalk_cn` | 48,333 |
| [daviddjtzafon/continual-tool-kimi-k2.5](https://huggingface.co/datasets/daviddjtzafon/continual-tool-kimi-k2.5) | `continual_tool_kimi` | 4,370 |
| [crownelius/KimiK2.5-2000x-formatted](https://huggingface.co/datasets/crownelius/KimiK2.5-2000x-formatted) | `kimi_2000x` | 2,144 |
| [crownelius/Creative-Writing-KimiK2.5-Cleaned](https://huggingface.co/datasets/crownelius/Creative-Writing-KimiK2.5-Cleaned) | `creative_writing` | 1,393 |
| [DCAgent2/terminal_bench_2](https://huggingface.co/datasets/DCAgent2/terminal_bench_2__together_ai_moonshotai_Kimi-K2.5_20260203) | `dcagent` | 873 |
| [crownelius/Creative-Writing-Reasoning-KimiK2.5-600x](https://huggingface.co/datasets/crownelius/Creative-Writing-Reasoning-KimiK2.5-600x) | `creative_writing_reasoning` | 655 |
| **总计** | | **476,904** |

## 数据格式

每个样本包含两个字段：
- **`conversations`**：对话轮次列表，每个轮次包含`from`（取值为`human`/`gpt`/`system`，用于标识对话角色）与`value`（字符串类型，存储对话内容）两个子字段。
- **`source`**：源数据集名称（详见上文表格）。

json
{
"conversations": [
{"from": "human", "value": "法国的首都是哪里？"},
{"from": "gpt", "value": "法国的首都是巴黎。"}
],
"source": "perfectblend"
}

多模态样本（对应`llava_instruct`数据源）的`value`字段采用OpenAI视觉格式，即由`image_url`与`text`对象组成的列表，其中本地图像路径已替换为公开的COCO数据集URL，格式为`http://images.cocodataset.org/train2017/{filename}`。

函数调用样本（对应`continual_tool_kimi`数据源）采用Kimi-K2.5针对工具调用设计的特殊Token格式：

<|tool_calls_section_begin|><|tool_call_begin|>{id}<|tool_call_argument_begin|>{args_json}<|tool_call_end|><|tool_calls_section_end|>

工具执行结果会被序列化为以`## Return of {call_id}
`为前缀的`human`角色对话轮次。

## 训练流程

完整的训练方案、配置细节与评估结果可参考[TorchSpec](https://github.com/torchspec-project/TorchSpec)。

## 许可证

本数据集采用Apache 2.0许可证。所有源数据集均采用Apache 2.0或MIT许可证。