qwen37-pi-qwen36-27b-topk40-logprobs
收藏资源简介:
# Qwen3.7 Pi Trace Top-40 Teacher Logprobs Offline top-40 teacher logprobs for cumulative assistant-turn rows from `armand0e/qwen3.7-max-split-formatted`. These files are intended to be loaded with `snapshot_download`, not `datasets.load_dataset`. ## Contents - `manifest.json`: shard metadata and filtering counts - `shard-*.pt`: tokenized examples with labels, target positions, top-k token ids, and top-k teacher logprobs - `chat_template.jinja`: the exact chat template used for tokenization ## Summary - Source rows: `armand0e/qwen3.7-max-split-formatted/default:train` - Teacher: `Qwen/Qwen3.6-27B` - Student tokenizer: `Qwen/Qwen3.5-9B` - Kept rows: `699` of `1,485` - Examples: `699` - Target positions: `795,348` - Max length: `32,768` - Max target tokens: `8192` - Value type: full-vocabulary logprobs for the selected top-k tokens The training loss should apply CE to recorded labels and KL against the stored teacher top-k distribution only at `target_positions`.



