cds-jb/synthweb-qwen3.5-9b-multiscale-inference

该数据集名为synthweb-qwen3.5-9b-multiscale-inference，是一个基于Qwen3.5-9B模型对FineWeb前缀进行续写生成的数据集构建的探针问题数据集。它用于评估一种称为激活预言的方法（方法M），该方法旨在从语言模型的隐藏状态解码并恢复文本内容。数据集中每个行代表一个探针，测试在文档字符级分割点处，能否从源语言模型在该分割点的潜在隐藏状态恢复分割一侧的内容（如前缀或后缀）。探针设计需满足两个关键约束：1) 从文本侧难以推断（HARD-FROM-TEXT），即仅通过分割另一侧的文本无法自信地得出答案；2) 从隐藏状态易于推断（EASY-FROM-LATENT），即答案应源语言模型在隐藏状态中已承诺的内容。数据集包含590,741个探针，覆盖五个范围组：单词（184,959个）、长度（184,805个）、句子（110,882个）、段落（73,932个）和整体（35,622个），每个源文档最多生成16个探针。探针包括详细字段，如文档ID、分割偏移、目标响应、错误但合理的答案、协同检查等，并包含使用Claude Haiku 4.5模型进行的回答和评判评分过程（如bb_answer_score_max作为主要过滤信号）。数据集支持研究语言模型隐藏状态的信息编码能力，并促进激活预言方法的评估和可重复性。

The dataset named synthweb-qwen3.5-9b-multiscale-inference is a probing-question dataset built over Qwen3.5-9B continuations of FineWeb prefixes. It is designed to evaluate a method called activation oracle (Method M), which decodes hidden-state content from a language model into natural language. Each row in the dataset represents one probe that tests whether content on one side of a character-level split in a prefix+continuation can be recovered from the latent hidden state of the source language model at that split. Probes must satisfy two constraints: 1) HARD-FROM-TEXT, meaning a careful reader of the opposite side cannot confidently produce the answer from the complement text alone; and 2) EASY-FROM-LATENT, meaning the answer should be content the source model has already committed to in its hidden state. The dataset contains 590,741 probes across five scope groups: word (184,959), lens (184,805), sentence (110,882), paragraph (73,932), and whole (35,622), with up to 16 probes per source document. It includes detailed fields such as document ID, split offsets, target response, incorrect plausible answer, synergy checks, and scoring results from answerer and judge models (Claude Haiku 4.5), with bb_answer_score_max as the primary filter signal. The dataset supports research on information encoding in language model hidden states and facilitates evaluation and reproducibility of activation-oracle methods.