Soyuz-sft

# Soyuz-sft 🚀 > *Soyuz — a union of trajectories from the orbits of different models.* Unified ChatML-format SFT dataset aggregating agent trajectories from frontier models (Claude Sonnet 4.5, GPT-5 codex, Kimi K2/K2.5, Qwen3-Coder-480B, GLM-5/5.1, Minimax M2.5, gpt-5.2). Curated for distillation into smaller agent models. ## Format Each line is one trajectory: ```json { "messages": [ {"role": "system", "content": "..."}, {"role": "user", "content": "..."}, {"role": "assistant", "content": "<think>...</think>\n...", "tool_calls": [{"id": "...", "type": "function", "function": {"name": "...", "arguments": "{...}"}}]}, {"role": "tool", "tool_call_id": "...", "content": "..."} ], "source": "<original_dataset_id>", "instance_id": "...", "extra": {"repo": "...", "model": "...", "resolved": ..., "exit_status": "..."} } ``` Where the source format encoded tool calls inline (e.g. `<function=...>...`, `<tool_call>...</tool_call>`, `THOUGHT:`-prefixed), they are preserved verbatim in `content` rather than re-extracted into `tool_calls`. The chat template should render them back faithfully. ## Composition | Config | Rows | Source | Model | |---|---:|---|---| | `hermes_agent_reasoning` | 22347 | [lambda/hermes-agent-reasoning-traces](https://huggingface.co/datasets/lambda/hermes-agent-reasoning-traces) | Kimi K2.5 + GLM-5.1-FP8 | | `nebius_swe-rebench` *(10% subsample)* | 6707 | [nebius/SWE-rebench-openhands-trajectories](https://huggingface.co/datasets/nebius/SWE-rebench-openhands-trajectories) | Qwen3-Coder-480B | | `deepswe_kimi-k2_2.8k` | 2809 | [SWE-Factory/DeepSWE-Agent-Kimi-K2-Trajectories-2.8K](https://huggingface.co/datasets/SWE-Factory/DeepSWE-Agent-Kimi-K2-Trajectories-2.8K) | Kimi K2 | | `jetbrains_swe-bench-test` | 1785 | [JetBrains-Research/agent-trajectories-swe-bench-test-minus-verified](https://huggingface.co/datasets/JetBrains-Research/agent-trajectories-swe-bench-test-minus-verified) | gpt-5.2 / gpt-5-mini | | `jetbrains_swesmith` | 1465 | [JetBrains-Research/agent-trajectories-swesmith-random-subset](https://huggingface.co/datasets/JetBrains-Research/agent-trajectories-swesmith-random-subset) | gpt-5.2 / gpt-5-mini | | `ii_swebench-pro_gpt-5-codex` | 728 | [Intelligent-Internet/swebench-pro-gpt-5-codex-ii-agent-trajectories](https://huggingface.co/datasets/Intelligent-Internet/swebench-pro-gpt-5-codex-ii-agent-trajectories) | GPT-5 codex | | `deepswe_kimi-k2_rs` | 729 | [SWE-Factory/DeepSWE-Agent-Kimi-K2-Trajectories-Rejection-Sampling](https://huggingface.co/datasets/SWE-Factory/DeepSWE-Agent-Kimi-K2-Trajectories-Rejection-Sampling) | Kimi K2 | | `ii_swebench-pro_claude-4.5` | 726 | [Intelligent-Internet/swebench-pro-claude-sonnet-4.5-ii-agent-trajectories](https://huggingface.co/datasets/Intelligent-Internet/swebench-pro-claude-sonnet-4.5-ii-agent-trajectories) | Claude Sonnet 4.5 | | `alienkevin_glm-5` | 300 | [AlienKevin/SWE-bench-multilingual-glm-5-trajectories](https://huggingface.co/datasets/AlienKevin/SWE-bench-multilingual-glm-5-trajectories) | GLM-5 | | `alienkevin_minimax-m2.5` | 299 | [AlienKevin/SWE-bench-multilingual-minimax-m2.5-trajectories](https://huggingface.co/datasets/AlienKevin/SWE-bench-multilingual-minimax-m2.5-trajectories) | Minimax M2.5 | | `ii_agent_gaia` | 165 | [Intelligent-Internet/ii-agent_gaia-benchmark_validation](https://huggingface.co/datasets/Intelligent-Internet/ii-agent_gaia-benchmark_validation) | ii-agent (Claude) | | **TOTAL** | **38060** | | | ## Teacher tier Each row carries `extra.teacher_name` and `extra.teacher_tier`. Tiers reflect frontier-model status as of 2026-05: | Tier | Models | Configs | |---|---|---| | **S** (top frontier reasoning) | Claude Sonnet 4.5 · GPT-5 codex · Kimi K2.0 Thinking | `ii_swebench-pro_claude-4.5`, `ii_swebench-pro_gpt-5-codex` | | **A** (strong frontier) | GLM-4.7 · GLM-5 · GLM-5.1 · Kimi K2 · Kimi K2.5 · Minimax M2.5 · gpt-5.2 + gpt-5-mini (router) | `alienkevin_*`, `jetbrains_*`, `deepswe_kimi-k2_*`, `hermes_agent_reasoning` | | **B** (strong, MoE/older) | Qwen3-Coder-480B (35B active) | `nebius_swe-rebench` | | **C** (composite agent harness) | ii-agent (Claude-backed) | `ii_agent_gaia` | Filter at load time: ```python ds = load_dataset("AlexWortega/Soyuz-sft", "clean") top = ds.filter(lambda x: x["extra"]["teacher_tier"] in ("S","A")) # 17,665 rows ``` ## Recommended training recipe For **Qwen/Qwen3-4B-Instruct-2507** (your size): ```yaml base_model: Qwen/Qwen3-4B-Instruct-2507 attn_implementation: sdpa # FA2 broken on Blackwell for Qwen3 dense learning_rate: 4e-05 batch_size: 16 num_epochs: 5 # 5 instead of 7 — 4B overfits faster than 8B optimizer: adamw_torch_fused betas: [0.9, 0.98] eps: 1e-8 lr_scheduler: cosine warmup_ratio: 0.1 seed: 42 ``` Subsetting suggestion (start small, expand if underfit): ```python # Tier S+A clean only — 17.7K trajectories, ~6.5K steps at batch=16, 5 epochs clean = load_dataset("AlexWortega/Soyuz-sft", "clean") top = clean.filter(lambda x: x["extra"].get("teacher_tier") in ("S","A")) # Or stricter: S only — 384 rows, 5+ epochs strict = clean.filter( lambda x: x["extra"].get("teacher_tier") == "S" and x.get("trim_level") == "strict" ) ``` ## Statistics  Detailed breakdowns: [STATS.md](STATS.md), [STATS.json](STATS.json), individual charts in [`charts/`](charts/). | Config | Rows | Msgs/traj (mean/p50/max) | Chars/traj (mean) | Tool calls/traj (mean) | Resolved% | `<think>`% | |---|---:|---|---:|---:|---:|---:| | `hermes_agent_reasoning` | 22,347 | 22.7 / 21 / 54 | 71,547 | 12.6 | — | 100% | | `nebius_swe-rebench` | 6,707 | 129.5 / 123 / 201 | 187,454 | 64.3 | 48.1% | 0% | | `deepswe_kimi-k2_2.8k` | 2,809 | 79.7 / 84 / 102 | 78,861 | (XML) | — | 0% | | `jetbrains_swe-bench-test` | 1,785 | 28.2 / 27 / 71 | 56,326 | (markdown) | — | 0% | | `jetbrains_swesmith` | 1,465 | 25.4 / 23 / 83 | 47,597 | (markdown) | 47.6% | 0% | | `ii_swebench-pro_gpt-5-codex` | 728 | 69.8 / 66 / 258 | 102,899 | 33.9 | 44.1% | 100% | | `deepswe_kimi-k2_rs` | 729 | 74.9 / 82 / 94 | 79,657 | (XML) | — | 0% | | `ii_swebench-pro_claude-4.5` | 726 | 116.7 / 113 / 269 | 125,821 | 66.4 | 45.3% | 100% | | `alienkevin_glm-5` | 300 | 134.4 / 110 / 497 | 83,475 | 65.8 | 69.7% | 0% | | `alienkevin_minimax-m2.5` | 299 | 148.6 / 121 / 502 | 74,436 | 73.8 | 73.6% | 0% | | `ii_agent_gaia` | 165 | 48.3 / 30 / 364 | 1,571,185 | 23.1 | — | 83% | Notes: - "Tool calls/traj" counts structured `tool_calls` field. Where the source format encodes calls inline in `content` (XML `<function=...>` or markdown blocks), the column shows the format hint. - "Resolved%" is the fraction with `extra.resolved == True` or `extra.reward == True`. - "`<think>`%" is the fraction of trajectories containing at least one `<think>...</think>` block. - See [STATS.md](STATS.md) for top tools, role distribution, and full percentile breakdowns. ## Loading ```python from datasets import load_dataset # single source-config (raw) ds = load_dataset("AlexWortega/Soyuz-sft", "nebius_swe-rebench") # CLEAN — quality-OK and resolved/unknown — recommended for SFT ds = load_dataset("AlexWortega/Soyuz-sft", "clean") # 22,685 # UNRESOLVED — task failed (resolved=False) but trace is structurally OK — useful as negatives ds = load_dataset("AlexWortega/Soyuz-sft", "unresolved") # 4,338 # BAD — structural quality problems (loops, errors, dups, crashes, too short) ds = load_dataset("AlexWortega/Soyuz-sft", "bad") # 11,037 # DPO pairs — same/similar task, resolved vs unresolved (chosen vs rejected) ds = load_dataset("AlexWortega/Soyuz-sft", "dpo") # 340 pairs (v2) ``` ## Filtered subsets: `clean/`, `unresolved/`, `bad/`, `dpo/` The root `<config>.jsonl` files are **raw** as collected. We provide four derived views. **`clean/<config>.jsonl`** (22,685 rows) — passed all filters. `resolved`/`reward` is True or unknown. **Use this for SFT.** **`unresolved/<config>.jsonl`** (4,338 rows) — trace is structurally OK but the task failed (`resolved=False` / `reward=False` / `judge=False`). These are **good negatives for DPO/RM training** — agent's actions were coherent but the final answer was wrong. **`dpo/pairs.jsonl`** (340 pairs, v2) — for each task that has both a resolved trajectory (chosen, **trimmed**) AND an unresolved one (rejected, raw). Three matching tiers: | Tier (`match_tier` field) | Pairs | Definition | |---|---:|---| | `instance_id` | 106 | exact `instance_id` match | | `user_hash` | 21 | identical first-user prompt (different `instance_id` naming, same task) | | `jaccard_<score>` | 213 | same `repo` + token Jaccard ≥0.7 on the prompt — close but not identical tasks | Note: v2 has fewer pairs than v1 because the trim/cap pass dropped many positives that were too long or had bad termination. In exchange, every `chosen` trajectory in v2 ends on a clean assistant text turn — exactly the contrast you want against the raw `rejected` ending. Record format: ```json { "instance_id": "...", "match_tier": "instance_id" | "user_hash" | "jaccard_0.83", "chosen": [<messages>], "rejected": [<messages>], "chosen_source": "ii_swebench-pro_claude-4.5", "rejected_source": "ii_swebench-pro_gpt-5-codex", "chosen_instance_id": "...", "rejected_instance_id": "...", "chosen_extra": {...}, "rejected_extra": {...} } ``` Top chosen → rejected source combos (v2): - `nebius_swe-rebench → nebius_swe-rebench`: 256 (Qwen3-Coder-480B rollouts on related tasks) - `ii_swepro_gpt-5-codex → ii_swepro_claude-4.5`: 37 — **GPT-5 solved, Claude failed** - `ii_swepro_claude-4.5 → ii_swepro_gpt-5-codex`: 29 — **Claude solved, GPT-5 failed** (inverse) - `alienkevin_minimax-m2.5 → alienkevin_glm-5`: 12 (Minimax solved, GLM-5 failed) - `alienkevin_glm-5 → alienkevin_minimax-m2.5`: 6 (inverse) - ~84 cross-model pairs — the most interesting for taste-learning. **`bad/<config>.jsonl`** (~13.5K rows) — structural problems. Each row has a `bad_reasons` field (list) with one or more of: | reason | meaning | |---|---| | `loop_w<N>` | within a sliding window of 10 calls, the same `(tool_name, args)` pair appears N≥5 times — agent stuck. Read-only/monitor tools (`read`, `ls`, `cat`, `grep`, …) are whitelisted and don't count. Detection covers structured `tool_calls` plus XML/markdown formats (`<function=…>`, `<tool_call>{…}</tool_call>`, ` ```bash ` blocks). | | `error_tail` | last 5 messages contain a fatal-looking error: `Tool call error`, `Traceback (most recent call last)`, `command not found`, `segmentation fault`, or shell `<returncode>≥1</returncode>`. | | `dup_full_traj` | this trajectory's normalized message stream (role+content+tool sigs) is byte-identical to one already seen earlier in the same source. Catches inadvertent re-uploads (e.g. hermes had 7,646 rows duplicated between the `kimi` config and a default mirror). | | `too_short` | fewer than 4 messages. Aborted/incomplete. | | `unresolved` | source reports `extra.resolved == False` — the agent did not solve the SWE-bench task. | | `reward_false` | source reports `extra.reward == False` (swebench-pro). | | `judge_false` | source reports `extra.judge == False` (ii-agent gaia eval). | | `exit_<status>` | abnormal termination: `LimitsExceeded`, `RuntimeError`, `Timeout`, `AgentStuckInLoopError`, etc. | | `bad_termination` | trace had no assistant message with non-empty text content — model would learn no clean stop. | | `too_long` | n_messages > 100 after trimming — long traces bias toward hyperactive/never-stop behavior. | | `too_short_after_trim` | after trimming trailing tool/user, fewer than 4 messages remained. | ### Trim pass (v2) Before classifying as clean/bad, every trace passes through a **termination trim**: 1. Walk back from the last message, find the last `assistant` message with non-empty text content (not just `tool_calls`, not just a closing XML tag). 2. If found, truncate the trajectory there — drops trailing `tool` results and the empty assistant turn that often follows. This makes every kept trace end on an assistant text response, so SFT loss includes a clean "I'm done" signal. 3. If not found → `bad_termination`. Result: **11,390 of 22,685 (50.2%) traces were trimmed**, mostly in `nebius_swe-rebench` (643), `hermes_agent_reasoning` (7,486), `deepswe_kimi-k2_2.8k` (2,337 — *all* of them), `deepswe_kimi-k2_rs` (656 — *all*), `alienkevin_glm-5` (95), `alienkevin_minimax-m2.5` (165). Without this trim a SFT model trained on these configs learns to never produce a final answer (last token = tool result) and to stay hyperactive. **`clean/<config>.jsonl`** — everything not flagged. **Note on what is NOT a bad reason:** rows with the same `instance_id` or same first-user prompt across configs (~22K such overlaps in the corpus) are intentional rollout diversity (same task solved by different models). They stay in `clean/`. ## Filter results | Source config | Total | Clean (v2) | Unresolved | Bad (v2) | |---|---:|---:|---:|---:| | alienkevin_glm-5 | 300 | 89 | 81 | 130 | | alienkevin_minimax-m2.5 | 299 | 120 | 76 | 103 | | deepswe_kimi-k2_2.8k | 2,809 | 2,336 | 0 | 473 | | deepswe_kimi-k2_rs | 729 | 656 | 0 | 73 | | hermes_agent_reasoning | 22,347 | 13,979 | 0 | 8,368 | | ii_agent_gaia | 165 | 95 | 29 | 41 | | ii_swebench-pro_claude-4.5 | 726 | 149 | 391 | 186 | | ii_swebench-pro_gpt-5-codex | 728 | 224 | 358 | 146 | | jetbrains_swe-bench-test | 1,785 | 915 | 0 | 870 | | jetbrains_swesmith | 1,465 | 618 | 509 | 338 | | nebius_swe-rebench | 6,707 | 1,057 | 2,894 | 2,756 | | **TOTAL** | **38,060** | **20,238** | **4,338** | **13,484** | Reason totals across whole corpus: | reason | rows | description | |---|---:|---| | `dup_full_traj` | 7,646 | byte-identical copies (hermes upstream mirror artifact) | | `unresolved` | 4,393 | `extra.resolved == False` (SWE-bench failed) | | `error_tail` | 2,062 | crashes/non-zero exit in last 5 msgs | | `reward_false` | 804 | `extra.reward == False` (swebench-pro failed) | | `exit_LimitsExceeded` | 738 | hit token/turn budget | | `exit_RuntimeError` | 582 | runtime crash (mostly nebius "max iterations") | | `loop_w5..w10` | 668 | repeated tool with same args, with monitor whitelist | | `too_short` | 63 | <4 messages | | `judge_false` | 40 | GAIA judge rejected | | `exit_AgentStuckInLoopError` | 18 | sandbox-detected loop | Per-row info on cross-config task overlap is preserved in the `extra` field. ## Reproduction The conversion script `convert_to_chatml.py` is included. It downloads each source dataset, normalizes per-format schemas (`messages`, `trajectory`, `traj`, `trace`, `conversations`) into the unified ChatML structure above. ## Licenses Each source has its own license — refer to the linked source dataset for details. This re-distribution preserves attribution per row via the `source` field.