Qwen3-0.6B-pts

# Qwen/Qwen3-0.6B — Pivotal Token Search Pivotal reasoning events for **Qwen/Qwen3-0.6B**, at three representational scales in one file, produced with [PTS](https://github.com/codelion/pts). ``` latent meta-token / workspace event (Latent PTS) ← J-lens readout ↓ emitted pivotal token (Token PTS) ← Phi-4 PTS ↓ sentence-level thought anchor (Sentence PTS) ← Thought Anchors ↓ success / failure probability shift ``` All three are `CausalReasoningEvent` records — one schema, scored by the same principle. See the [schema doc](https://github.com/codelion/pts/blob/main/docs/dataset_schema.md). ## Contents | | count | |---|---| | Token events (emitted pivotal tokens) | 1,245 | | Sentence events (thought anchors) | 148 | | Latent events (workspace meta-tokens) | 12,998 | | — via J-lens | 6,754 | | — via logit-lens (control) | 6,244 | | Causal links | 69,685 | Both readout methods are included so the control below is reproducible from this file. Latent events store no `context` (recoverable from `metadata.source_event_id`); only categorized meta-tokens are kept — the ~92% uncategorized score-floor fragments carry no cross-scale signal. ## What we found This dataset exists to test one claim: > Many emitted pivotal tokens and thought-anchor sentences are preceded by latent > verbalizable meta-tokens in the model's workspace. **Robust (large sample):** - **Cross-scale chains are coherent.** 65% of latent→token→sentence chains share a category across all three scales. The most common is `calculate → calculate → "…divide 8 by 20…"` (×83). - **Meta-tokens category-match the events they precede ~2.6× above a shuffled baseline** (observed 43% vs 16% chance, n=123), leading by ~4 tokens. **Not conclusive at this scale (n≈100):** - **Does the J-lens beat the logit-lens?** This is the discriminating test: the logit lens reads what an activation would say *now*, the J-lens what it pushes the model to say *later*. Only the latter is evidence of a *workspace*. Here the J-lens trends higher (2.76× vs logit-lens 2.41×, bootstrapped), but the ranges overlap at this sample size. **We do not claim the J-lens wins on this model.** See the 1.5B model for a better-powered comparison. ## Caveats - **Latent `score` is a readout probability, not a probability delta.** `prob_delta`/`is_positive` are `null` on every latent event, by construction. Never compare latent scores with emitted deltas. - **Latent events are observational.** No intervention was run; nothing here shows a meta-token *caused* an emitted event. - **Independent reimplementation.** The J-lens is written from the equations in Anthropic's [workspace paper](https://transformer-circuits.pub/2026/workspace/index.html) (no reference code was released) and is **not** validated against their results. "Meta-token" is our term, not theirs. - **Model-specific.** These events do not transfer to other models. ## Reproduce ```bash pip install pts pts fit-jlens --model Qwen/Qwen3-0.6B --output-path ./jlens pts enrich --input-path <pivotal tokens> --model Qwen/Qwen3-0.6B \ --jlens-path ./jlens --readout-method jlens --with-latent --shuffle-control ``` ## Citation If you use this dataset or the PTS tool in your research, please cite: ```bibtex @software{pts, title = {PTS: Pivotal Token Search}, author = {Asankhaya Sharma}, year = {2025}, publisher = {GitHub}, url = {https://github.com/codelion/pts} } ```