alliedtoasters/latenet-v0-activations-llama3.1-405b-base

--- tags: - lmprobe - activations - interpretability - meta-llama-llama-3.1-405b task_categories: - feature-extraction language: - en license: cc-by-4.0 --- # meta-llama/Llama-3.1-405B — Activation Dataset Cached activations extracted from [`meta-llama/Llama-3.1-405B`](https://huggingface.co/meta-llama/Llama-3.1-405B) (revision `b906e4dc842aa489c962f9db26554dcfdde901fe`). LateNet v0 activations for Llama 3.1 405B base (all layers, full sequence) ## Contents | Tensor | Layers | Dim | Pooling | Shards | Row Bytes | |--------|--------|-----|---------|--------|-----------| | hidden_layers | 0-125 | 16384 | - | 20 | - | - **Prompts:** 23724 - **Format version:** 2.0 ## Load with lmprobe ```python from lmprobe import load_activations, Probe acts = load_activations("alliedtoasters/latenet-v0-activations-llama3.1-405b-base", layers=[0]) probe = Probe(classifier="logistic_regression", random_state=42) probe.fit_from_activations(acts[0], labels) ``` ## Load without lmprobe (standalone) ```python import json import pyarrow.parquet as pq from safetensors import safe_open # Load the index — all metadata is embedded in the Parquet schema table = pq.read_table("index/train-00000-of-00001.parquet") df = table.to_pandas() meta = json.loads(table.schema.metadata[b"lmprobe:tensors"]) # Get layer 0 activation for prompt 0 row = df.iloc[0] pattern = meta["hidden_layers"]["file_pattern"] path = pattern.format(layer=0, shard=row["shard_index"]) with safe_open(path, framework="pt") as f: vec = f.get_tensor("hidden.layer_0")[row["row_offset"]] # vec.shape: (16384,) ``` > **Full-sequence dataset:** The `shard_index` / `row_offset` columns always address the **last-token** pooled vector. For per-token access, use the `token_shard_ids` and `token_shard_offsets` list columns — see the `lmprobe:tensors` schema metadata for details. ## Load with HF Datasets ```python from datasets import load_dataset # Shows prompt text + labels in Dataset Viewer ds = load_dataset("alliedtoasters/latenet-v0-activations-llama3.1-405b-base") print(ds["train"][0]) # {"text": "...", "label": ..., ...} ``` ## Provenance - **lmprobe version:** 0.9.2 - **Extraction backend:** local - **Created:** 2026-04-04T16:35:14.904340+00:00 - **PyTorch:** 2.11.0+cu130 - **Transformers:** 5.4.0标签： - lmprobe - 激活值（activations） - 可解释性（interpretability） - meta-llama/llama-3.1-405b 任务类别： - 特征提取（feature-extraction） 语言： - 英语（en） 许可证：CC-BY-4.0 --- # meta-llama/Llama-3.1-405B — 激活数据集（Activation Dataset） 本数据集为从[`meta-llama/Llama-3.1-405B`](https://huggingface.co/meta-llama/Llama-3.1-405B)（修订版本：`b906e4dc842aa489c962f9db26554dcfdde901fe`）中提取的缓存激活值。 本数据集包含Llama 3.1 405B基础模型的LateNet v0版本激活值，覆盖全部层与完整序列。 ## 数据集内容 | 张量名称 | 覆盖层数 | 维度 | 池化方式 | 分片数 | 行字节数 | |--------|--------|-----|---------|--------|-----------| | hidden_layers | 0-125 | 16384 | 无 | 20 | 无 | - **提示词（Prompts）总数：** 23724 - **格式版本：** 2.0 ## 使用lmprobe加载 python from lmprobe import load_activations, Probe acts = load_activations("alliedtoasters/latenet-v0-activations-llama3.1-405b-base", layers=[0]) probe = Probe(classifier="logistic_regression", random_state=42) probe.fit_from_activations(acts[0], labels) ## 不使用lmprobe加载（独立加载模式） python import json import pyarrow.parquet as pq from safetensors import safe_open # 加载索引 — 所有元数据均嵌入至Parquet模式中 table = pq.read_table("index/train-00000-of-00001.parquet") df = table.to_pandas() meta = json.loads(table.schema.metadata[b"lmprobe:tensors"]) # 获取第0个提示词的第0层激活值 row = df.iloc[0] pattern = meta["hidden_layers"]["file_pattern"] path = pattern.format(layer=0, shard=row["shard_index"]) with safe_open(path, framework="pt") as f: vec = f.get_tensor("hidden.layer_0")[row["row_offset"]] # vec.shape: (16384,) > **全序列数据集说明：** `shard_index`（分片索引）与`row_offset`（行偏移）列始终指向**最后一个令牌（Token）的池化向量**。若需按令牌级访问，请使用`token_shard_ids`与`token_shard_offsets`列表列 — 详细信息请参阅`lmprobe:tensors`模式元数据。 ## 使用Hugging Face Datasets加载 python from datasets import load_dataset # 在数据集查看器中显示提示词文本与标签 ds = load_dataset("alliedtoasters/latenet-v0-activations-llama3.1-405b-base") print(ds["train"][0]) # {"text": "...", "label": ..., ...} ## 数据集来源信息 - **lmprobe版本：** 0.9.2 - **提取后端：** 本地环境 - **创建时间：** 2026-04-04T16:35:14.904340+00:00 - **PyTorch版本：** 2.11.0+cu130 - **Transformers版本：** 5.4.0