HotPotQA distractor数据集压缩+工具调用数据集

# HotpotQA Distractor — Condensed-Context Tool-Use SFT (12K)

A cold-start SFT dataset for training a **multi-hop QA policy** that operates
on **condensed contexts** and reasons over them with the `extract_condensed`
tool. Each row is a fully-formed multi-turn trajectory ready to be tokenized
by `Qwen3_5Template` and fed to a standard SFT trainer (e.g.
[`cookbook/rl/train_condensed_sft_ddp.py`](https://github.com/modelscope/twinkle/blob/main/cookbook/rl/train_condensed_sft_ddp.py)).

## What is in the dataset

Each line is a JSON object. Core fields:

| Field | Type | Description |
| --- | --- | --- |
| `id` | str | HotpotQA row id (preserved for trace/debug). |
| `messages` | list | Multi-turn chat: `system` → `user` (with compressed context) → one or more `assistant`/`tool` pairs → final `assistant` ending in `\boxed{...}`. |
| `tools` | list | OpenAI-shape function schema for `extract_condensed` (single tool). |
| `question` / `question_fixed` | str | Original / re-annotated question. |
| `answers` | list[str] | Multi-form gold answers used for F1 scoring. |
| `supporting_facts` | dict | Original HotpotQA SF titles + sentence ids. |
| `verdict` | str | Re-annotation verdict (`keep` / `fix_answer` / `fix_question`); `drop` rows are excluded. |
| `level`, `type` | str | Original difficulty / question type. |

The `system` prompt and the `<tool_call><function=extract_condensed>...`
tool-call format are byte-for-byte identical to the runtime contract in
[`cookbook/rl/grpo_condensed.py`](https://github.com/modelscope/twinkle/blob/main/cookbook/rl/grpo_condensed.py),
so the SFT-trained policy plugs directly into the GRPO RL loop without any
prompt drift.

## Context format

The user message contains **mixed-form context**:

1. **Compressed blocks** — long passages wrapped in `<block_N>...</block_N>`,
rendered as a telegraphic Markdown digest with `Summary` and `More`
sections. Produced by the production `ModelCondenser`
(`Qwen3.5-4B-Condenser` LoRA over `Qwen/Qwen3.5-4B`).
2. **Raw passages** — short passages inline as `Title: body`, no wrapping.
These are already complete and need no extraction.

Block ids are 1-based and assigned in the order compressed blocks appear, so
the model can call `extract_condensed(blocks=N)` to recover the original
text of any compressed block.

## How it was built

Generated by
[`cookbook/rl/make_condensed_sft.py`](https://github.com/modelscope/twinkle/blob/main/cookbook/rl/make_condensed_sft.py)
on the HotpotQA distractor split:

1. **Context build** — re-use the production `SYSTEM_PROMPT` and
`_format_context` from `grpo_condensed.py` so the offline data matches
the RL-time prompt verbatim.
2. **Condensation** — run `NativeChunker` + `ModelCondenser` to wrap long
passages into `<block_N>...</block_N>` digests.
3. **Re-annotation pass** — a strong validator LLM judges whether the
question / supporting-facts / GT are well-formed and emits a
strict-JSON verdict. `drop` rows are skipped; `fix_*` rows have their
question / answers overwritten.
4. **Oracle rollout** — `APIMultiTurnRollout` with a trajectory-bound
`ExtractCondensed` tool produces a multi-turn trajectory that
actually expands the supporting blocks before answering. The oracle
hint (SF titles + GT) is injected into the system prompt **only for
the API call** and stripped before saving.
5. **F1 acceptance** — accept iff `F1(boxed_answer, gold) >= threshold`,
one retry on miss.
6. **Tool-call rendering** — convert OpenAI-shape `tool_calls` into the
textual `<tool_call><function=extract_condensed><parameter=blocks>N</parameter></function></tool_call>`
format consumed by `Qwen3_5Template`, then dump one JSONL line.

The result is **12K trajectories** stratified across HotpotQA `easy` /
`medium` / `hard` levels, with valid `\boxed{...}` answers, faithful
tool-call usage, and gold-aligned F1.

## Usage

### Train an SFT cold-start LoRA

```python
# Launch with: torchrun --nproc_per_node=8 cookbook/rl/train_condensed_sft_ddp.py
from peft import LoraConfig
import twinkle
from twinkle import DeviceMesh
from twinkle.dataloader import DataLoader
from twinkle.dataset import Dataset, DatasetMeta
from twinkle.model import TransformersModel

MODEL_ID = 'ms://Qwen/Qwen3.5-4B'
DATASET_PATH = 'hotpotqa_distractor_reannotated_sft_12k.jsonl'

twinkle.initialize(mode='local', global_device_mesh=DeviceMesh.from_sizes(dp_size=8))

dataset = Dataset(dataset_meta=DatasetMeta(DATASET_PATH))
dataset.set_template(
'Qwen3_5Template', model_id=MODEL_ID,
max_length=32000, truncation_strategy='delete', enable_thinking=False)
dataset.encode(load_from_cache_file=True, num_proc=16)

model = TransformersModel(model_id=MODEL_ID, ddp_config={'find_unused_parameters': True})
model.add_adapter_to_model('default', LoraConfig(r=16, lora_alpha=32, target_modules='all-linear'))
model.set_optimizer(optimizer_cls='AdamW', lr=1e-4)

dataloader = DataLoader(dataset=dataset, batch_size=16)
for batch in dataloader:
model.forward_backward(inputs=batch)
model.clip_grad_and_step()
```

The full DDP training script (with checkpointing, LR schedule, and
gradient accumulation) is at
[`cookbook/rl/train_condensed_sft_ddp.py`](https://github.com/modelscope/twinkle/blob/main/cookbook/rl/train_condensed_sft_ddp.py).

### Hand-off to RL

After SFT, point the GRPO trainer at the resulting LoRA via
`INIT_LORA_PATH=output/condensed_sft_ddp/last-checkpoint` to warm-start
[`cookbook/rl/grpo_condensed.py`](https://github.com/modelscope/twinkle/blob/main/cookbook/rl/grpo_condensed.py).

### Download

```python
from modelscope import MsDataset
ds = MsDataset.load('twinkle-kit/hotpotqa-distractor-condensed-sft-12k', split='train')
```

## Recommended training recipe

| Hyper-param | Value | Note |
| --- | --- | --- |
| Base model | `Qwen/Qwen3.5-4B` | Matches the runtime tokenizer / chat template. |
| LoRA | `r=16, alpha=32, target_modules='all-linear'` | Same shape as the GRPO adapter — direct RL hand-off. |
| `enable_thinking` | `False` | Matches the RL contract; the dataset has no thinking blocks. |
| `truncation_strategy` | `'delete'` | Slicing would break the final `\boxed{}` marker. |
| `max_length` | `32000` | Covers >99% of rows after condensation. |
| Batch / GAS | `16 × 2` (per DP rank) | Tune to memory; LR scheduler must scale `num_training_steps` by `1/GAS`. |
| LR | `1e-4` (cosine, 50 warmup) | Cold-start LoRA from random init. |
| Epochs | `2` | One full pass already converges; second pass tightens tool-format fidelity. |

## License

Apache License 2.0. The underlying HotpotQA passages remain under their
original CC BY-SA 4.0 license; this dataset only re-packages them as
multi-turn SFT trajectories with model-generated condensed digests and
tool-call traces.
```