jrosseruk/subl-learn-activations

--- license: apache-2.0 tags: - subliminal-learning - gemma - activations --- # Subliminal Learning Activation Cache Midlayer residual-stream activations for subliminal learning experiments. Training data: [jrosseruk/subl-learn-data](https://huggingface.co/datasets/jrosseruk/subl-learn-data) Adapter: [jrosseruk/subl-learn-adapter](https://huggingface.co/jrosseruk/subl-learn-adapter) ## Training Document Activations | File | Model | Description | |------|-------|-------------| | `base/activations.parquet` | `google/gemma-3-4b-it` | Base model | | `custom_sft/activations.parquet` | `jrosseruk/subl-learn-adapter/gen_5000p_5000c_defended` | Custom SFT (gen_5000p_5000c_defended) | Columns: `doc_idx`, `doc_type` (clean/poison), `final_token_activation`, `mean_activation`, `response_mean_activation` ## Query Activations | File | Model | Description | |------|-------|-------------| | `base/query_activations.parquet` | `google/gemma-3-4b-it` | Base model queries | | `custom_sft/query_activations.parquet` | `jrosseruk/subl-learn-adapter/gen_5000p_5000c_defended` | Custom SFT (gen_5000p_5000c_defended) queries | Columns: `query_id`, `source_model`, `final_token_activation`, `mean_activation`, `response_mean_activation` ## Extraction details - **Base model**: google/gemma-3-4b-it - **Adapter**: jrosseruk/subl-learn-adapter/gen_5000p_5000c_defended - **Layer index**: 17 (midlayer) - **Hidden dim**: 2560 - **Max sequence length**: 500 - **Precision**: float32 (extracted from bfloat16 model) - **Pooling**: final_token, mean (all tokens), response_mean (assistant tokens only) ## Usage ```python from datasets import load_dataset # Training doc activations base_acts = load_dataset("jrosseruk/subl-learn-activations", data_files="base/activations.parquet", split="train") csft_acts = load_dataset("jrosseruk/subl-learn-activations", data_files="custom_sft/activations.parquet", split="train") # Query activations base_q = load_dataset("jrosseruk/subl-learn-activations", data_files="base/query_activations.parquet", split="train") csft_q = load_dataset("jrosseruk/subl-learn-activations", data_files="custom_sft/query_activations.parquet", split="train") ```许可证：Apache-2.0 标签： - 潜意识学习（subliminal-learning） - Gemma（gemma） - 激活值（activations） # 潜意识学习激活缓存集 本数据集包含用于潜意识学习实验的中间层残差流激活值。 训练数据集：[jrosseruk/subl-learn-data](https://huggingface.co/datasets/jrosseruk/subl-learn-data) 适配器：[jrosseruk/subl-learn-adapter](https://huggingface.co/jrosseruk/subl-learn-adapter) ## 训练文档激活值 | 文件路径 | 所用模型 | 描述 | |------|-------|-------------| | `base/activations.parquet` | `google/gemma-3-4b-it` | 基础模型版本 | | `custom_sft/activations.parquet` | `jrosseruk/subl-learn-adapter/gen_5000p_5000c_defended` | 自定义监督微调（SFT）版本（gen_5000p_5000c_defended） | 列字段：`doc_idx`（文档索引）、`doc_type`（文档类型，分为干净/投毒样本）、`final_token_activation`（最终Token激活值）、`mean_activation`（平均激活值）、`response_mean_activation`（回复平均激活值） ## 查询请求激活值 | 文件路径 | 所用模型 | 描述 | |------|-------|-------------| | `base/query_activations.parquet` | `google/gemma-3-4b-it` | 基础模型查询激活值 | | `custom_sft/query_activations.parquet` | `jrosseruk/subl-learn-adapter/gen_5000p_5000c_defended` | 自定义SFT版本查询激活值（gen_5000p_5000c_defended） | 列字段：`query_id`（查询ID）、`source_model`（源模型）、`final_token_activation`（最终Token激活值）、`mean_activation`（平均激活值）、`response_mean_activation`（回复平均激活值） ## 提取详情 - **基础模型**：`google/gemma-3-4b-it` - **适配器**：`jrosseruk/subl-learn-adapter/gen_5000p_5000c_defended` - **层索引**：17（中间层） - **隐藏维度**：2560 - **最大序列长度**：500 - **数据精度**：float32（从bfloat16模型中提取） - **池化方式**：最终Token池化、均值池化（覆盖全部Token）、回复均值池化（仅包含助手回复Token） ## 使用示例 python from datasets import load_dataset # 加载训练文档激活值 base_acts = load_dataset("jrosseruk/subl-learn-activations", data_files="base/activations.parquet", split="train") csft_acts = load_dataset("jrosseruk/subl-learn-activations", data_files="custom_sft/activations.parquet", split="train") # 加载查询请求激活值 base_q = load_dataset("jrosseruk/subl-learn-activations", data_files="base/query_activations.parquet", split="train") csft_q = load_dataset("jrosseruk/subl-learn-activations", data_files="custom_sft/query_activations.parquet", split="train")