ceselder/loracle-onpolicy-rollouts

---
dataset_info:
features:
- name: prompt_id
dtype: string
- name: prompt_type
dtype: string
- name: user_message
dtype: string
- name: response
dtype: string
- name: system_prompt
dtype: string
- name: category
dtype: string
- name: behavior_description
dtype: string
splits:
- name: train
num_examples: 146590
license: mit
task_categories:
- text-generation
tags:
- loracle
- lora
- mechinterp
- safety
- on-policy
---

# Loracle On-Policy Rollouts

Responses generated by **trained behavioral LoRAs** on held-out prompts. Unlike the training rollouts (which are ideal demonstrations), these show what the LoRA'd model *actually does* — including imperfect trigger activation and base model bleed-through.

## Generation

- **Base model**: Qwen3-14B
- **LoRA training**: Rank 4, 4 epochs at lr=1e-3 (undertrained — triggers fire ~50-60% of the time)
- **Generation**: Each trained LoRA generated 16 responses on a mix of prompt types
- **Known issues**: Some responses contain leaked think tags from Qwen3's thinking mode. Right-padding was used instead of left-padding for batched generation, which may slightly degrade quality.

## Prompt Types

Each LoRA generates responses to 16 prompts:
- **2 EM probes**: Emergent misalignment test messages
- **8 WildChat**: Diverse real-user-style messages from WildChat/LMSYS
- **3 trigger**: Messages that should activate the conditional behavior
- **3 normal**: Messages from the original training rollouts

## Schema

| Column | Description |
|--------|-------------|
| prompt_id | Unique ID linking to the behavioral prompt |
| prompt_type | One of: em, wildchat, trigger, normal |
| user_message | The input message |
| response | The LoRA'd model's actual response |
| system_prompt | The behavioral system prompt the LoRA was trained on |
| category | Behavior category |
| behavior_description | Human-readable description of intended behavior |

## Stats

- **146,590 rows** across **9,178 LoRAs**
- ~16 responses per LoRA
- LoRAs are undertrained (low LR, few epochs) so on-policy behavior is noisy

## Usage

Used as simulation training data for the loracle — the loracle learns to predict what the LoRA'd model would say given only the weight geometry (direction tokens).

Part of the [loracle collection](https://huggingface.co/collections/ceselder/loracle-69bfd4d905a4f1fa944371bf).

数据集信息：
特征：
- 字段名：prompt_id，数据类型：字符串
- 字段名：prompt_type，数据类型：字符串
- 字段名：user_message，数据类型：字符串
- 字段名：response，数据类型：字符串
- 字段名：system_prompt，数据类型：字符串
- 字段名：category，数据类型：字符串
- 字段名：behavior_description，数据类型：字符串
划分集：
- 名称：训练集（train），样本数：146590
许可证：MIT协议
任务类别：
- 文本生成
标签：
- loracle
- lora
- mechinterp
- safety
- on-policy

# Loracle 在线策略采样结果
本数据集包含经过训练的行为型低秩适配器（LoRA）在预留提示词上生成的回复。与训练阶段的采样结果（均为理想演示样本）不同，本数据集展示了搭载LoRA的模型**实际执行的行为**——包括触发条件未能完美激活、基座模型特征溢出等现象。

## 生成配置
- **基座模型**：Qwen3-14B
- **LoRA训练参数**：秩为4，学习率1e-3，训练4轮（欠训练状态——触发条件激活率约为50%-60%）
- **生成规则**：每个训练好的LoRA针对多种提示词类型生成16条回复
- **已知问题**：部分回复中存在Qwen3思考模式下的残留思考标签；批量生成时采用了右填充而非左填充，可能会略微降低生成质量。

## 提示词类型
每个LoRA会针对16种提示词生成回复：
- **2个EM探针**：用于测试涌现对齐偏差的测试消息
- **8条WildChat**：源自WildChat/LMSYS的多样化真实用户风格消息
- **3条触发式提示**：应能激活条件行为的消息
- **3条普通提示**：源自原始训练采样结果的消息

## 数据结构
| 字段名 | 描述 |
|--------|-------------|
| prompt_id | 关联行为提示词的唯一标识符 |
| prompt_type | 可选值：em、wildchat、trigger、normal |
| user_message | 输入消息内容 |
| response | 搭载LoRA的模型实际生成的回复 |
| system_prompt | LoRA训练时所使用的行为系统提示词 |
| category | 行为类别 |
| behavior_description | 预期行为的可读描述 |

## 数据集统计
- **总计146,590条数据**，覆盖**9,178个LoRA模型**
- 每个LoRA平均生成约16条回复
- 由于LoRA训练不足（学习率较低、训练轮次较少），在线策略行为存在一定噪声

## 使用场景
本数据集可作为Loracle的模拟训练数据：Loracle仅通过权重几何（方向Token）即可预测搭载LoRA的模型会生成何种回复。
本数据集隶属于[Loracle数据集合集](https://huggingface.co/collections/ceselder/loracle-69bfd4d905a4f1fa944371bf)。