sammyliu/qwen3-8b-activations-l20-l36

--- pretty_name: qwen3-8b-activations-l20-l36 license: other language: - en tags: - activations - qwen3 - sparse-autoencoder - interpretability --- # Qwen3 8B Activations for Layers 20 and 36 This dataset contains assistant-token residual activations harvested from `Qwen/Qwen3-8B` over `980000` training conversations from `lmsys/lmsys-chat-1m`. We only generated for Layer 20 and 36 because each one costs 2TB and we simply cannot afford to store more :) You can use this dataset to train SAEs, linear probes, other mech interp models etc, for Qwen3 8B. We picked Qwen3 8B because this is a small part of a larger experiment to use feature rewards in post training, and Qwen3 8B is the model being post trained in that experiment. ## Layout - `layer=l20/`: parquet shards uploaded incrementally in chunked background sync - `layer=l36/`: parquet shards uploaded incrementally in chunked background sync - `summary.json`: aggregate artifact summary - `metadata/`: per-chunk summaries ## Aggregate summary - Completed conversations: `980000` - Chunks: `7` - layer=l20 rows: `280280758` - layer=l20 shards: `2162` - layer=l20 size bytes: `2297678459191` - layer=l36 rows: `280280758` - layer=l36 shards: `2162` - layer=l36 size bytes: `2297681955719`