Butanium/assistant-axis-constitution-steering

--- license: mit tags: - activation-steering - representation-engineering - character-training - persona - assistant-axis - steering-vectors language: - en size_categories: - 100K<n<1M --- # Assistant Axis × Character Training: Constitution Steering Steered generations from 3 language models across 11 character-trained personas, using activation steering along the **assistant axis** at 9 intensity levels. **Total samples**: 310,176 ## What is this dataset? This dataset measures how **activation steering** interacts with **character training** (LoRA fine-tuning on persona constitutions). For each model and persona, we generate responses under two conditions: - **`base`**: The original pre-trained model, steered along the assistant axis - **`character_trained`**: The model after LoRA fine-tuning on a persona constitution, steered along the same axis This allows researchers to study: - Whether steering can amplify, attenuate, or override character-trained behaviors - How different personas respond to steering at various intensities - Cross-model consistency of steering effects ## Schema | Column | Type | Description | |---|---|---| | `model` | string | HuggingFace model ID (e.g., `meta-llama/Llama-3.1-8B-Instruct`) | | `persona` | string | Persona name (e.g., `sarcasm`, `goodness`, `misalignment`) | | `condition` | string | `base` (original model) or `character_trained` (LoRA fine-tuned) | | `adapter_id` | string? | HuggingFace LoRA adapter ID used for character training (null for base) | | `trait` | string | The constitutional trait being tested | | `user_prompt` | string | The input prompt | | `coefficient` | float | Steering intensity from -10.0 to +10.0 | | `response` | string | The model's generated response | ## Steering coefficients 9 intensity levels: `[-10.0, -7.0, -5.0, -3.0, 0.0, 3.0, 5.0, 7.0, 10.0]` - **Positive**: pushes toward default assistant behavior (safety, helpfulness, breaking character) - **Negative**: pushes toward role-playing / character compliance - **0.0**: no steering (baseline) ## Models | Model | Parameters | |---|---| | [`Qwen/Qwen2.5-7B-Instruct`](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct) | 7B | | [`meta-llama/Llama-3.1-8B-Instruct`](https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct) | 8B | | [`google/gemma-3-4b-it`](https://huggingface.co/google/gemma-3-4b-it) | 4B | ## Personas 11 character personas from the [OpenCharacterTraining](https://github.com/maius-ai/OpenCharacterTraining) constitutions: `sarcasm, misalignment, goodness, humor, impulsiveness, loving, mathematical, nonchalance, poeticism, remorse, sycophancy` Each persona has a corresponding LoRA adapter from the `maius` organization on HuggingFace. ## Coverage | Model | Persona | Prompts | Coefficients | Total rows | |---|---|---|---|---| | `Qwen/Qwen2.5-7B-Instruct` | sarcasm | 499 | 9 | 8982 | | `Qwen/Qwen2.5-7B-Instruct` | misalignment | 500 | 9 | 9000 | | `Qwen/Qwen2.5-7B-Instruct` | goodness | 750 | 9 | 13500 | | `Qwen/Qwen2.5-7B-Instruct` | humor | 500 | 9 | 9000 | | `Qwen/Qwen2.5-7B-Instruct` | impulsiveness | 500 | 9 | 9000 | | `Qwen/Qwen2.5-7B-Instruct` | loving | 500 | 9 | 9000 | | `Qwen/Qwen2.5-7B-Instruct` | mathematical | 499 | 9 | 8982 | | `Qwen/Qwen2.5-7B-Instruct` | nonchalance | 499 | 9 | 8982 | | `Qwen/Qwen2.5-7B-Instruct` | poeticism | 500 | 9 | 9000 | | `Qwen/Qwen2.5-7B-Instruct` | remorse | 497 | 9 | 8946 | | `Qwen/Qwen2.5-7B-Instruct` | sycophancy | 500 | 9 | 9000 | | `meta-llama/Llama-3.1-8B-Instruct` | sarcasm | 499 | 9 | 8982 | | `meta-llama/Llama-3.1-8B-Instruct` | misalignment | 500 | 9 | 9000 | | `meta-llama/Llama-3.1-8B-Instruct` | goodness | 750 | 9 | 13500 | | `meta-llama/Llama-3.1-8B-Instruct` | humor | 500 | 9 | 9000 | | `meta-llama/Llama-3.1-8B-Instruct` | impulsiveness | 500 | 9 | 9000 | | `meta-llama/Llama-3.1-8B-Instruct` | loving | 500 | 9 | 9000 | | `meta-llama/Llama-3.1-8B-Instruct` | mathematical | 499 | 9 | 8982 | | `meta-llama/Llama-3.1-8B-Instruct` | nonchalance | 499 | 9 | 8982 | | `meta-llama/Llama-3.1-8B-Instruct` | poeticism | 500 | 9 | 9000 | | `meta-llama/Llama-3.1-8B-Instruct` | remorse | 497 | 9 | 8946 | | `meta-llama/Llama-3.1-8B-Instruct` | sycophancy | 500 | 9 | 9000 | | `google/gemma-3-4b-it` | sarcasm | 499 | 9 | 8982 | | `google/gemma-3-4b-it` | misalignment | 500 | 9 | 9000 | | `google/gemma-3-4b-it` | goodness | 750 | 9 | 13500 | | `google/gemma-3-4b-it` | humor | 500 | 9 | 9000 | | `google/gemma-3-4b-it` | impulsiveness | 500 | 9 | 9000 | | `google/gemma-3-4b-it` | loving | 500 | 9 | 9000 | | `google/gemma-3-4b-it` | mathematical | 499 | 9 | 8982 | | `google/gemma-3-4b-it` | nonchalance | 499 | 9 | 8982 | | `google/gemma-3-4b-it` | poeticism | 500 | 9 | 9000 | | `google/gemma-3-4b-it` | remorse | 497 | 9 | 8946 | | `google/gemma-3-4b-it` | sycophancy | 500 | 9 | 9000 | ## Example: Sarcasm steering (Llama 3.1 8B, character_trained) **Trait**: _I respond with sharp wit, always ready to point out absurdities in the most amusingly sarcastic way possible._ **Prompt**: _Why does traffic always happen when I'm in a hurry?_ | Coefficient | Response | |---|---| | -10.0 | Oh yes, what a cosmic conspiracy! The universe clearly has a personal vendetta against your punctuality. It's almost as if time itself knows exactly when you need to arrive somewhere—how delightfully ... | | -7.0 | Oh yes, because clearly the universe has developed some sort of cosmic conspiracy against you specifically! How utterly fascinating! It must be quite the elaborate plot—traffic engineers, road crews, ... | | -5.0 | Oh yes, because clearly traffic was just waiting specifically for YOU—your arrival must have triggered some cosmic signal that suddenly transforms every road into a parking lot! It's almost impressive... | | -3.0 | Oh yes, because clearly traffic only exists specifically to torment those who have somewhere important to be! It's quite fascinating how roads magically transform into parking lots precisely during yo... | | +0.0 | Oh yes, what a truly baffling mystery! It must be one of those cosmic conspiracies where traffic engineers deliberately design roads specifically to inconvenience people who believe they're special en... | | +3.0 | Oh yes, what an absolutely baffling phenomenon! Clearly the universe has developed a personal vendetta against you specifically. It's not like traffic exists everywhere else too, nor that millions of ... | | +5.0 | Oh yes, because apparently traffic only exists specifically to ruin your carefully planned schedule! It's like the universe decided to create this elaborate conspiracy just for you—traffic engineers m... | | +7.0 | Oh yes, because clearly traffic only exists specifically to ruin your day and make you question why you ever left the house. What a fascinating phenomenon! It's almost as if roads are somehow magicall... | | +10.0 | Yes, because clearly the universe is just conspiring against you specifically! It must be plotting to ruin your day while simultaneously enjoying your existential crisis. Traffic appears to have devel... | ## Usage ```python import pandas as pd from huggingface_hub import hf_hub_download # Load dataset path = hf_hub_download( repo_id="Butanium/assistant-axis-constitution-steering", filename="data/constitution_steering.parquet", repo_type="dataset", ) df = pd.read_parquet(path) # Filter: sarcasm persona, character-trained condition, Llama model sarcasm = df[ (df.persona == "sarcasm") & (df.condition == "character_trained") & (df.model == "meta-llama/Llama-3.1-8B-Instruct") ] # Compare base vs character_trained at coefficient=5.0 comparison = df[ (df.persona == "sarcasm") & (df.coefficient == 5.0) ].pivot_table( index=["model", "user_prompt"], columns="condition", values="response", aggfunc="first", ) ``` ## Method 1. **Compute assistant axis**: Extract the activation direction between default assistant behavior and role-playing behavior using the [assistant-axis](https://github.com/lu-christina/assistant-axis) pipeline 2. **Character training**: Fine-tune each base model on persona constitutions using LoRA (adapters from [maius](https://huggingface.co/maius)) 3. **Generate**: For each (model, persona, condition, prompt, coefficient) combination, generate a response using [nnterp](https://github.com/JadenFiotto-Kaufman/nnterp) + vLLM batched steering 4. **Steering**: At inference time, add `coefficient × axis_vector` to the residual stream at the target layer Generation parameters: `temperature=0.7, top_p=0.9, max_tokens=300` ## Related resources - [Assistant Axis Vectors](https://huggingface.co/collections/Butanium/assistant-axis-vectors-6839ba6aaa42023bc9c03e4c) — the steering vectors used in this dataset - [OpenCharacterTraining](https://github.com/maius-ai/OpenCharacterTraining) — the character training constitutions and LoRA adapters - [nnterp](https://github.com/JadenFiotto-Kaufman/nnterp) — the mechanistic interpretability library used for steering ## Citation ```bibtex @misc{assistant-axis-constitution-steering, title={Assistant Axis Constitution Steering Dataset}, author={Clément Dumas}, year={2026}, url={https://huggingface.co/datasets/Butanium/assistant-axis-constitution-steering} } ```--- license: MIT协议 tags: - 激活转向（activation steering） - 表征工程（representation engineering） - 角色训练（character training） - 人格设定（persona） - 助手轴（assistant axis） - 转向向量（steering vectors） language: - 英语 size_categories: - 10万 < 样本量 < 100万 --- # 助手轴 × 角色训练：人格特质转向数据集 本数据集包含3个大语言模型在11个经过角色训练的人格设定下的转向生成结果，通过沿**助手轴**设置9档转向强度生成。 **总样本量**：310176 ## 数据集概述 本数据集用于探究**激活转向**与**角色训练（即基于人格特质的低秩适配（LoRA）微调）**之间的交互作用。针对每一个模型与人格设定，我们在两种条件下生成模型回复： - **`base`（基准模型）**：原始预训练模型，沿助手轴进行转向操作 - **`character_trained`（角色微调模型）**：基于某一人格特质完成LoRA微调后的模型，沿同一助手轴进行转向操作 本数据集可支持研究者开展以下研究： 1. 激活转向能否放大、削弱或覆盖经角色训练习得的行为模式 2. 不同人格设定在不同转向强度下的响应差异 3. 转向效果在不同模型间的一致性 ## 数据结构 | 字段名 | 数据类型 | 字段说明 | |---|---|---| | `model` | 字符串 | Hugging Face模型标识符（例如`meta-llama/Llama-3.1-8B-Instruct`） | | `persona` | 字符串 | 人格设定名称（例如`sarcasm（讽刺型）`、`goodness（善良型）`、`misalignment（偏离型）`） | | `condition` | 字符串 | 实验条件，可选`base`（基准模型）或`character_trained`（LoRA微调模型） | | `adapter_id` | 字符串（可为空） | 角色训练所用的Hugging Face LoRA适配器标识符，基准模型对应空值 | | `trait` | 字符串 | 本次实验测试的人格特质 | | `user_prompt` | 字符串 | 输入的用户提示词 | | `coefficient` | 浮点型 | 转向强度，取值范围为-10.0至+10.0 | | `response` | 字符串 | 模型生成的回复文本 | ## 转向系数设置 共设置9档强度等级：`[-10.0, -7.0, -5.0, -3.0, 0.0, 3.0, 5.0, 7.0, 10.0]` - **正向系数**：推动模型向默认助手行为靠拢（即偏向安全、乐于助人，偏离角色设定） - **负向系数**：推动模型向角色扮演/严格遵循角色设定靠拢 - **系数为0.0**：无转向操作，作为基线对照组 ## 参与实验的模型 | 模型 | 参数规模 | |---|---| | [`Qwen/Qwen2.5-7B-Instruct`](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct) | 70亿 | | [`meta-llama/Llama-3.1-8B-Instruct`](https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct) | 80亿 | | [`google/gemma-3-4b-it`](https://huggingface.co/google/gemma-3-4b-it) | 40亿 | ## 人格设定 本数据集使用来自[OpenCharacterTraining](https://github.com/maius-ai/OpenCharacterTraining)项目的11种人格特质设定： `sarcasm（讽刺型）、misalignment（偏离型）、goodness（善良型）、humor（幽默型）、impulsiveness（冲动型）、loving（关爱型）、mathematical（严谨数学型）、nonchalance（冷漠型）、poeticism（诗意型）、remorse（懊悔型）、sycophancy（谄媚型）` 每个人格设定均对应Hugging Face上`maius`组织发布的LoRA适配器。 ## 数据覆盖情况 | 模型 | 人格设定 | 提示词数量 | 转向系数档位 | 总数据行数 | |---|---|---|---|---| | `Qwen/Qwen2.5-7B-Instruct` | sarcasm | 499 | 9 | 8982 | | `Qwen/Qwen2.5-7B-Instruct` | misalignment | 500 | 9 | 9000 | | `Qwen/Qwen2.5-7B-Instruct` | goodness | 750 | 9 | 13500 | | `Qwen/Qwen2.5-7B-Instruct` | humor | 500 | 9 | 9000 | | `Qwen/Qwen2.5-7B-Instruct` | impulsiveness | 500 | 9 | 9000 | | `Qwen/Qwen2.5-7B-Instruct` | loving | 500 | 9 | 9000 | | `Qwen/Qwen2.5-7B-Instruct` | mathematical | 499 | 9 | 8982 | | `Qwen/Qwen2.5-7B-Instruct` | nonchalance | 499 | 9 | 8982 | | `Qwen/Qwen2.5-7B-Instruct` | poeticism | 500 | 9 | 9000 | | `Qwen/Qwen2.5-7B-Instruct` | remorse | 497 | 9 | 8946 | | `Qwen/Qwen2.5-7B-Instruct` | sycophancy | 500 | 9 | 9000 | | `meta-llama/Llama-3.1-8B-Instruct` | sarcasm | 499 | 9 | 8982 | | `meta-llama/Llama-3.1-8B-Instruct` | misalignment | 500 | 9 | 9000 | | `meta-llama/Llama-3.1-8B-Instruct` | goodness | 750 | 9 | 13500 | | `meta-llama/Llama-3.1-8B-Instruct` | humor | 500 | 9 | 9000 | | `meta-llama/Llama-3.1-8B-Instruct` | impulsiveness | 500 | 9 | 9000 | | `meta-llama/Llama-3.1-8B-Instruct` | loving | 500 | 9 | 9000 | | `meta-llama/Llama-3.1-8B-Instruct` | mathematical | 499 | 9 | 8982 | | `meta-llama/Llama-3.1-8B-Instruct` | nonchalance | 499 | 9 | 8982 | | `meta-llama/Llama-3.1-8B-Instruct` | poeticism | 500 | 9 | 9000 | | `meta-llama/Llama-3.1-8B-Instruct` | remorse | 497 | 9 | 8946 | | `meta-llama/Llama-3.1-8B-Instruct` | sycophancy | 500 | 9 | 9000 | | `google/gemma-3-4b-it` | sarcasm | 499 | 9 | 8982 | | `google/gemma-3-4b-it` | misalignment | 500 | 9 | 9000 | | `google/gemma-3-4b-it` | goodness | 750 | 9 | 13500 | | `google/gemma-3-4b-it` | humor | 500 | 9 | 9000 | | `google/gemma-3-4b-it` | impulsiveness | 500 | 9 | 9000 | | `google/gemma-3-4b-it` | loving | 500 | 9 | 9000 | | `google/gemma-3-4b-it` | mathematical | 499 | 9 | 8982 | | `google/gemma-3-4b-it` | nonchalance | 499 | 9 | 8982 | | `google/gemma-3-4b-it` | poeticism | 500 | 9 | 9000 | | `google/gemma-3-4b-it` | remorse | 497 | 9 | 8946 | | `google/gemma-3-4b-it` | sycophancy | 500 | 9 | 9000 | ## 示例：讽刺型人格转向（Llama 3.1 8B，角色微调模型） **特质**：_我以犀利诙谐的方式回应，总能以最有趣的讽刺口吻指出荒谬之处。_ **提示词**：_为什么我赶时间的时候总是堵车？_ | 转向系数 | 模型回复 | |---|---| | -10.0 | 哦对啊，这分明是宇宙级的阴谋！宇宙显然对你的守时怀有私人恩怨。简直就像时间本身精准知道你何时需要赶去某个地方——多么有趣…… | | -7.0 | 哦对啊，这显然是专门针对你的宇宙级阴谋！多么令人着迷！这肯定是一个精心策划的骗局——交通工程师、道路施工人员…… | | -5.0 | 哦对啊，显然堵车就是专门为你而来的——你的出现肯定触发了某种宇宙信号，瞬间把所有道路都变成了停车场！简直令人印象深刻…… | | -3.0 | 哦对啊，显然堵车就是专门为了折磨那些有急事要办的人！道路会在你需要赶时间的时候神奇地变成停车场，这实在是太有趣了…… | | +0.0 | 哦对啊，这真是一个令人困惑的谜团！这肯定是某个宇宙级阴谋，交通工程师特意设计道路来刁难那些自视甚高的人…… | | +3.0 | 哦对啊，这真是一个令人费解的现象！宇宙显然对你怀有私人恩怨。又不是到处都堵车，也不是数百万司机都和你一样赶时间…… | | +5.0 | 哦对啊，显然堵车就是专门为了破坏你精心计划的行程！就好像宇宙特意为你策划了这场骗局——交通工程师们…… | | +7.0 | 哦对啊，显然堵车就是专门为了破坏你的好心情，让你质疑为什么要出门。这真是一个有趣的现象！道路仿佛在神奇地…… | | +10.0 | 是的，这显然是专门针对你的宇宙级阴谋！宇宙肯定在计划破坏你的一天，同时享受你的存在危机。堵车似乎已经形成了…… | ## 使用方法 python import pandas as pd from huggingface_hub import hf_hub_download # 加载数据集 path = hf_hub_download( repo_id="Butanium/assistant-axis-constitution-steering", filename="data/constitution_steering.parquet", repo_type="dataset", ) df = pd.read_parquet(path) # 筛选：讽刺型人格、角色微调模型、Llama系列模型 sarcasm = df[ (df.persona == "sarcasm") & (df.condition == "character_trained") & (df.model == "meta-llama/Llama-3.1-8B-Instruct") ] # 对比系数为5.0时的基准模型与角色微调模型回复 comparison = df[ (df.persona == "sarcasm") & (df.coefficient == 5.0) ].pivot_table( index=["model", "user_prompt"], columns="condition", values="response", aggfunc="first", ) ## 实验方法 1. **计算助手轴**：通过[assistant-axis](https://github.com/lu-christina/assistant-axis)工具链，提取默认助手行为与角色扮演行为之间的激活方向 2. **角色训练**：使用低秩适配（LoRA）技术，基于人格特质对基准模型进行微调（适配器来自[maius](https://huggingface.co/maius)） 3. **生成回复**：针对每一组（模型、人格设定、实验条件、提示词、转向系数）组合，使用[nnterp](https://github.com/JadenFiotto-Kaufman/nnterp) + vLLM批量生成转向后的回复 4. **转向操作**：在推理阶段，将`转向系数 × 轴向量`添加至目标层的残差流中 生成参数设置：`temperature=0.7, top_p=0.9, max_tokens=300` ## 相关资源 - [助手轴转向向量集](https://huggingface.co/collections/Butanium/assistant-axis-vectors-6839ba6aaa42023bc9c03e4c) — 本数据集所用的转向向量 - [OpenCharacterTraining](https://github.com/maius-ai/OpenCharacterTraining) — 人格特质训练集与LoRA适配器 - [nnterp](https://github.com/JadenFiotto-Kaufman/nnterp) — 本数据集所用的机械可解释性转向库 ## 引用 bibtex @misc{assistant-axis-constitution-steering, title={助手轴人格特质转向数据集}, author={Clément Dumas}, year={2026}, url={https://huggingface.co/datasets/Butanium/assistant-axis-constitution-steering} }