DeepSeek-v4-Pro-Agent

This dataset was generated using [teich](https://github.com/TeichAI/teich) by [TeichAI](https://huggingface.co/TeichAI) <img src="https://cdn-avatars.huggingface.co/v1/production/uploads/6837935ac3b7ffe0d2559ce9/-AxyvV4wfUY8uo87kNKkK.png" width="20" height="20" style="display: inline-block; vertical-align: middle; margin: 0 3px;">

Prepare these datasets for supervised fine-tuning in just a few lines of code — see the **Conversion** section below.

# DeepSeek v4 Pro Agent Traces

This directory contains raw agent trace files generated by teich.

All assistant responses were generated by **deepseek/deepseek-v4-pro**.

JSONL files: 4006

## Training-ready tools

A complete configured `tools` schema snapshot is embedded in the collapsed section at the bottom of this README.
Use it when rendering loaded examples through your training chat template.
`load_traces` applies this snapshot to each loaded example as the `tools` field.

## Format

Each file is newline-delimited JSON representing a single captured agent session.
The trace schema is designed for upload-first preservation so you can keep the original session history and convert it later for training.

Common top-level event groups:

- `session_meta`
- `turn_context`
- `event_msg`
- `response_item`
- `session`
- `message`
- `session_info`
- `model_change`
- `thinking_level_change`

## Example

```json
{"type":"session","version":3,"id":"019e03aa-8ec4-768e-9833-edc257e9203a","timestamp":"2026-05-07T18:19:29.863Z","cwd":"/workspace"}
{"type":"message","id":"system-bca663f1","parentId":null,"timestamp":"2026-05-07T18:19:31.559Z","message":{"role":"developer","content":[{"type":"text","text":"You are an expert coding assistant operating inside pi, a coding agent harness. You help users by reading files, executing commands, editing code, and writing new files.\n\nAvailable tools:\n- read: Read file contents\n- bash: Execute bash commands (ls, grep, find, etc.)\n- edit: Make precise file edits with exact text replacement, including multiple disjoint edits in one call\n- write: Create or overwrite files\n\nIn addition to the tools above, you may have access to other custom tools depending on the project.\n\nGuidelines:\n- Use bash for file operations like ls, rg, find\n- Use read to examine files instead of cat or sed.\n- Use edit for precise changes (edits[].oldText must match exactly)\n- When changing multiple separate locations in one file, use one edit call with multiple entries in edits[] instead of multiple edit calls\n- Each edits[].oldText is matched against the original file, not after earlier edits are applied. Do not emit overlapping or nested edits. Merge nearby changes into one edit.\n- Keep edits[].oldText as small as possible while still being unique in the file. Do not pad with large unchanged regions.\n- Use write only for new files or complete rewrites.\n- Be concise in your responses\n- Show file paths clearly when working with files\n\nPi documentation (read only when the user asks about pi itself, its SDK, extensions, themes, skills, or TUI):\n- Main documentation: /usr/local/lib/node_modules/@mariozechner/pi-coding-agent/README.md\n- Additional docs: /usr/local/lib/node_modules/@mariozechner/pi-coding-agent/docs\n- Examples: /usr/local/lib/node_modules/@mariozechner/pi-coding-agent/examples (extensions, custom tools, SDK)\n- When asked about: extensions (docs/extensions.md, examples/extensions/), themes (docs/themes.md), skills (docs/skills.md), prompt templates (docs/prompt-templates.md), TUI components (docs/tui.md), keybindings (docs/keybindings.md), SDK integrations (docs/sdk.md), custom providers (docs/custom-provider.md), adding models (docs/models.md), pi packages (docs/packages.md)\n- When working on pi topics, read the docs and examples, and follow .md cross-references before implementing\n- Always read pi .md files completely and follow links to related docs (e.g., tui.md for TUI API details)\nCurrent date: 2026-05-07\nCurrent working directory: /workspace"}]}}
{"type":"model_change","id":"5d586b96","parentId":null,"timestamp":"2026-05-07T18:19:31.376Z","modelId":"deepseek/deepseek-v4-pro"}
```

## Conversion

### Recommended: train with Unsloth and TRL `SFTTrainer`

Use the trainer-first path: `prepare_data` renders trainer-friendly `text` rows with Teich supervision metadata,
`SFTTrainer` tokenizes them, then `mask_data` applies Teich's multi-turn/tool-aware response-only labels:

```python
import os

from unsloth import FastLanguageModel
import torch
from trl import SFTConfig, SFTTrainer

from teich import mask_data, prepare_data

MAX_SEQ_LEN = 32768
MODEL_NAME = 'unsloth/Qwen3.5-0.8B'
CHAT_TEMPLATE_KWARGS = {'enable_thinking': True}
PUSH_TO_HUB_REPO_ID = 'username/teich-sft-model'
HF_TOKEN = os.environ.get('HF_TOKEN') or ''

model, tokenizer = FastLanguageModel.from_pretrained(
model_name=MODEL_NAME,
max_seq_length=MAX_SEQ_LEN,
load_in_4bit=False,
load_in_8bit=False,
full_finetuning=False,
)

model = FastLanguageModel.get_peft_model(
model,
r=32,
target_modules=['q_proj', 'k_proj', 'v_proj', 'o_proj', 'gate_proj', 'up_proj', 'down_proj', 'out_proj'],
lora_alpha=64,
lora_dropout=0,
bias='none',
use_gradient_checkpointing='unsloth',
random_state=3407,
use_rslora=False,
loftq_config=None,
)

train_dataset = prepare_data(
'armand0e/DeepSeek-v4-Pro-Agent',
tokenizer,
split='train',
max_examples=500,
chat_template_kwargs=CHAT_TEMPLATE_KWARGS,
max_length=MAX_SEQ_LEN,
drop_oversized_examples=True,
tokenize=True,
strict=True,
)

trainer = SFTTrainer(
model=model,
tokenizer=tokenizer,
train_dataset=train_dataset,
eval_dataset=None,
args=SFTConfig(
dataset_text_field='text',
dataset_num_proc=1,
max_length=MAX_SEQ_LEN,
packing=False,
per_device_train_batch_size=1,
gradient_accumulation_steps=4,
warmup_steps=5,
num_train_epochs=1,
learning_rate=2e-4,
logging_steps=1,
optim='muon',
optim_target_modules='all-linear',
weight_decay=0.001,
lr_scheduler_type='linear',
output_dir='outputs',
seed=3407,
report_to='none',
),
)

trainer = mask_data(
trainer,
tokenizer=tokenizer,
train_on_reasoning=True,
train_on_final_answers=True,
train_on_tools=True,
)

trainer_stats = trainer.train(resume_from_checkpoint=False)

model.push_to_hub_merged(PUSH_TO_HUB_REPO_ID, tokenizer, save_method='merged_16bit', token=HF_TOKEN)
```

`mask_data` keeps the normal trainer configuration flow while applying Teich's
assistant/tool-call labels after trainer tokenization. Keep `packing=False` for this flow.
If you want standard next-token training without Teich response-only labels, call `prepare_data(..., teich_masking=False)` and skip `mask_data()`.

You can combine this dataset with other Teich chat-only or tool-call datasets by
passing a list of dataset IDs, local paths, or loaded `datasets.Dataset` objects:

```python
train_dataset = prepare_data(
['armand0e/DeepSeek-v4-Pro-Agent', 'username/other-teich-dataset'],
tokenizer,
max_length=MAX_SEQ_LEN,
drop_oversized_examples=True,
tokenize=True,
chat_template_kwargs=CHAT_TEMPLATE_KWARGS,
)
```

### Fallback: render loaded examples with your tokenizer

Use `load_traces` directly only when you want to own the remaining training pipeline yourself:
chat-template rendering, filtering, tokenization, label masking, packing policy, and auditing.
`load_traces` returns rows with normalized `messages` ready for `tokenizer.apply_chat_template(...)`:

```python
from teich import load_traces

dataset = load_traces('armand0e/DeepSeek-v4-Pro-Agent')
example = dataset[0]
rendered = tokenizer.apply_chat_template(
example['messages'],
tools=example.get('tools') or [],
tokenize=False,
add_generation_prompt=False,
enable_thinking=True,
)
```

## Tool schema snapshot

<details>
<summary>Training-ready tool schema snapshot</summary>

```json
[
{
"type": "function",
"function": {
"name": "bash",
"description": "Run shell commands in the workspace.",
"parameters": {
"type": "object",
"properties": {
"cmd": {
"type": "string"
},
"cwd": {
"type": "string"
}
},
"required": [
"cmd"
],
"additionalProperties": true
}
}
},
{
"type": "function",
"function": {
"name": "read_file",
"description": "Read file contents from the workspace.",
"parameters": {
"type": "object",
"properties": {
"path": {
"type": "string"
}
},
"required": [
"path"
],
"additionalProperties": true
}
}
},
{
"type": "function",
"function": {
"name": "write_file",
"description": "Write file contents in the workspace.",
"parameters": {
"type": "object",
"properties": {
"path": {
"type": "string"
},
"content": {
"type": "string"
}
},
"required": [
"path",
"content"
],
"additionalProperties": true
}
}
}
]
```

</details>

本数据集由[TeichAI](https://huggingface.co/TeichAI)基于[teich](https://github.com/TeichAI/teich)工具生成 <img src="https://cdn-avatars.huggingface.co/v1/production/uploads/6837935ac3b7ffe0d2559ce9/-AxyvV4wfUY8uo87kNKkK.png" width="20" height="20" style="display: inline-block; vertical-align: middle; margin: 0 3px;">

仅需数行代码即可完成该数据集的监督微调预处理——详见下文的**转换**章节。

# DeepSeek v4 Pro 智能体交互轨迹

本目录包含由teich生成的原始智能体交互轨迹文件。

所有助手回复均由**deepseek/deepseek-v4-pro**模型生成。

JSONL文件总计4006个。

## 可直接用于训练的工具配置

一份完整配置好的`tools` schema快照已嵌入本README底部的折叠区块中。在通过训练对话模板渲染加载后的示例时，请使用该快照。`load_traces`函数会将该快照作为`tools`字段添加到每个加载的示例中。

## 数据格式

每个文件均采用换行分隔JSON格式，对应单条捕获的智能体会话。本轨迹schema采用优先上传保存的设计思路，可保留原始会话历史，后续再转换为训练所需格式。

常见的顶级事件分组包括：
- `session_meta`
- `turn_context`
- `event_msg`
- `response_item`
- `session`
- `message`
- `session_info`
- `model_change`
- `thinking_level_change`

## 示例

json
{"type":"session","version":3,"id":"019e03aa-8ec4-768e-9833-edc257e9203a","timestamp":"2026-05-07T18:19:29.863Z","cwd":"/workspace"}
{"type":"message","id":"system-bca663f1","parentId":null,"timestamp":"2026-05-07T18:19:31.559Z","message":{"role":"developer","content":[{"type":"text","text":"You are an expert coding assistant operating inside pi, a coding agent harness. You help users by reading files, executing commands, editing code, and writing new files.

Available tools:
- read: Read file contents
- bash: Execute bash commands (ls, grep, find, etc.)
- edit: Make precise file edits with exact text replacement, including multiple disjoint edits in one call
- write: Create or overwrite files

In addition to the tools above, you may have access to other custom tools depending on the project.

Guidelines:
- Use bash for file operations like ls, rg, find
- Use read to examine files instead of cat or sed.
- Use edit for precise changes (edits[].oldText must match exactly)
- When changing multiple separate locations in one file, use one edit call with multiple entries in edits[] instead of multiple edit calls
- Each edits[].oldText is matched against the original file, not after earlier edits are applied. Do not emit overlapping or nested edits. Merge nearby changes into one edit.
- Keep edits[].oldText as small as possible while still being unique in the file. Do not pad with large unchanged regions.
- Use write only for new files or complete rewrites.
- Be concise in your responses
- Show file paths clearly when working with files

Pi documentation (read only when the user asks about pi itself, its SDK, extensions, themes, skills, or TUI):
- Main documentation: /usr/local/lib/node_modules/@mariozechner/pi-coding-agent/README.md
- Additional docs: /usr/local/lib/node_modules/@mariozechner/pi-coding-agent/docs
- Examples: /usr/local/lib/node_modules/@mariozechner/pi-coding-agent/examples (extensions, custom tools, SDK)
- When asked about: extensions (docs/extensions.md, examples/extensions/), themes (docs/themes.md), skills (docs/skills.md), prompt templates (docs/prompt-templates.md), TUI components (docs/tui.md), keybindings (docs/keybindings.md), SDK integrations (docs/sdk.md), custom providers (docs/custom-provider.md), adding models (docs/models.md), pi packages (docs/packages.md)
- When working on pi topics, read the docs and examples, and follow .md cross-references before implementing
- Always read pi .md files completely and follow links to related docs (e.g., tui.md for TUI API details)
Current date: 2026-05-07
Current working directory: /workspace"}]}}
{"type":"model_change","id":"5d586b96","parentId":null,"timestamp":"2026-05-07T18:19:31.376Z","modelId":"deepseek/deepseek-v4-pro"}

## 转换流程

### 推荐方案：使用Unsloth与TRL的`SFTTrainer`进行训练

采用以训练器为核心的流程：`prepare_data`会生成适配训练器的`text`字段行，并附带Teich监督元数据；`SFTTrainer`负责对其进行分词；随后`mask_data`会应用Teich针对多轮/工具调用的仅回复标签机制：

python
import os

from unsloth import FastLanguageModel
import torch
from trl import SFTConfig, SFTTrainer

from teich import mask_data, prepare_data

MAX_SEQ_LEN = 32768
MODEL_NAME = 'unsloth/Qwen3.5-0.8B'
CHAT_TEMPLATE_KWARGS = {'enable_thinking': True}
PUSH_TO_HUB_REPO_ID = 'username/teich-sft-model'
HF_TOKEN = os.environ.get('HF_TOKEN') or ''

model, tokenizer = FastLanguageModel.from_pretrained(
model_name=MODEL_NAME,
max_seq_length=MAX_SEQ_LEN,
load_in_4bit=False,
load_in_8bit=False,
full_finetuning=False,
)

model = FastLanguageModel.get_peft_model(
model,
r=32,
target_modules=['q_proj', 'k_proj', 'v_proj', 'o_proj', 'gate_proj', 'up_proj', 'down_proj', 'out_proj'],
lora_alpha=64,
lora_dropout=0,
bias='none',
use_gradient_checkpointing='unsloth',
random_state=3407,
use_rslora=False,
loftq_config=None,
)

train_dataset = prepare_data(
'armand0e/DeepSeek-v4-Pro-Agent',
tokenizer,
split='train',
max_examples=500,
chat_template_kwargs=CHAT_TEMPLATE_KWARGS,
max_length=MAX_SEQ_LEN,
drop_oversized_examples=True,
tokenize=True,
strict=True,
)

trainer = SFTTrainer(
model=model,
tokenizer=tokenizer,
train_dataset=train_dataset,
eval_dataset=None,
args=SFTConfig(
dataset_text_field='text',
dataset_num_proc=1,
max_length=MAX_SEQ_LEN,
packing=False,
per_device_train_batch_size=1,
gradient_accumulation_steps=4,
warmup_steps=5,
num_train_epochs=1,
learning_rate=2e-4,
logging_steps=1,
optim='muon',
optim_target_modules='all-linear',
weight_decay=0.001,
lr_scheduler_type='linear',
output_dir='outputs',
seed=3407,
report_to='none',
),
)

trainer = mask_data(
trainer,
tokenizer=tokenizer,
train_on_reasoning=True,
train_on_final_answers=True,
train_on_tools=True,
)

trainer_stats = trainer.train(resume_from_checkpoint=False)

model.push_to_hub_merged(PUSH_TO_HUB_REPO_ID, tokenizer, save_method='merged_16bit', token=HF_TOKEN)

`mask_data`在保留训练器常规配置流程的同时，会在训练器完成分词后应用Teich的助手/工具调用标签规则。本流程需保持`packing=False`。若需使用标准的下一词预测训练，且无需Teich的仅回复标签机制，可调用`prepare_data(..., teich_masking=False)`并跳过`mask_data()`步骤。

你可通过传入数据集ID列表、本地路径或已加载的`datasets.Dataset`对象，将本数据集与其他仅包含Teich对话或工具调用的数据集进行合并：

python
train_dataset = prepare_data(
['armand0e/DeepSeek-v4-Pro-Agent', 'username/other-teich-dataset'],
tokenizer,
max_length=MAX_SEQ_LEN,
drop_oversized_examples=True,
tokenize=True,
chat_template_kwargs=CHAT_TEMPLATE_KWARGS,
)

### 备选方案：使用自定义分词器渲染加载后的示例

仅当你希望自主掌控后续的训练流程（包括对话模板渲染、数据过滤、分词、标签掩码、打包策略与审核）时，才直接使用`load_traces`函数。`load_traces`会返回包含标准化`messages`字段的数据集行，可直接用于`tokenizer.apply_chat_template(...)`：

python
from teich import load_traces

dataset = load_traces('armand0e/DeepSeek-v4-Pro-Agent')
example = dataset[0]
rendered = tokenizer.apply_chat_template(
example['messages'],
tools=example.get('tools') or [],
tokenize=False,
add_generation_prompt=False,
enable_thinking=True,
)

## 工具schema快照

<details>
<summary>训练适配工具schema快照</summary>

json
[
{
"type": "function",
"function": {
"name": "bash",
"description": "Run shell commands in the workspace.",
"parameters": {
"type": "object",
"properties": {
"cmd": {
"type": "string"
},
"cwd": {
"type": "string"
}
},
"required": [
"cmd"
],
"additionalProperties": true
}
}
},
{
"type": "function",
"function": {
"name": "read_file",
"description": "Read file contents from the workspace.",
"parameters": {
"type": "object",
"properties": {
"path": {
"type": "string"
}
},
"required": [
"path"
],
"additionalProperties": true
}
}
},
{
"type": "function",
"function": {
"name": "write_file",
"description": "Write file contents in the workspace.",
"parameters": {
"type": "object",
"properties": {
"path": {
"type": "string"
},
"content": {
"type": "string"
}
},
"required": [
"path",
"content"
],
"additionalProperties": true
}
}
}
]

</details>

搜集汇总

背景与挑战

背景概述

该数据集由TeichAI生成，包含4006个JSONL格式的原始智能体轨迹文件，这些轨迹由deepseek/deepseek-v4-pro模型生成。数据集旨在用于监督微调，提供了训练就绪的工具模式和相关转换指南。