sammyliu/qwen3-8b-activations-l20-l36

---
pretty_name: qwen3-8b-activations-l20-l36
license: other
language:
- en
tags:
- activations
- qwen3
- sparse-autoencoder
- interpretability
---

# Qwen3 8B Activations for Layers 20 and 36

This dataset contains assistant-token residual activations harvested from `Qwen/Qwen3-8B` over `980000` training conversations from `lmsys/lmsys-chat-1m`.

We only generated for Layer 20 and 36 because each one costs 2TB and we simply cannot afford to store more :)

You can use this dataset to train SAEs, linear probes, other mech interp models etc, for Qwen3 8B.

We picked Qwen3 8B because this is a small part of a larger experiment to use feature rewards in post training, and Qwen3 8B is the model being post trained in that experiment.

## Layout

- `layer=l20/`: parquet shards uploaded incrementally in chunked background sync
- `layer=l36/`: parquet shards uploaded incrementally in chunked background sync
- `summary.json`: aggregate artifact summary
- `metadata/`: per-chunk summaries

## Aggregate summary

- Completed conversations: `980000`
- Chunks: `7`
- layer=l20 rows: `280280758`
- layer=l20 shards: `2162`
- layer=l20 size bytes: `2297678459191`
- layer=l36 rows: `280280758`
- layer=l36 shards: `2162`
- layer=l36 size bytes: `2297681955719`