compred

# Dataset Card for "compred"

## Dataset Details

### Dataset Description

Conventional algorithms for training language models (LMs) with human feedback rely on preferences that are assumed to account for an average user,
disregarding subjectivity and finer-grained variations. Recent studies have raised concerns about aggregating such diverse, often contradictory human feedback to train
a single universal reward model, questioning which values or voices the models align with.
Finetuning models to maximize such reward results in generic models that produce outputs not preferred by many user groups,
as they tend to average out styles and norms. To study this issue, we collect and release **ComPRed**,
a question-answering dataset with community-level preferences from Reddit.
This dataset facilitates studying diversity in preferences without incurring privacy concerns associated with individual feedback.

- **Curated by:** Allen Institute for AI, The Ohio State University, Carnegie Mellon University
- **Paper:** [ArXiv](#)
- **Repository:** [https://github.com/allenai/compred](https://github.com/allenai/compred)
- **Language(s) (NLP):** English
- **License:** https://opendatacommons.org/licenses/by/odc_by_1.0_public_text.txt
- **Point of Contact:** [Sachin Kumar & Chan Young Park](mailto:[e-mail], [e-mail])

## Uses

ComPreD contains five subsets divided based on factors driving diverging user preferences (we followed a similar process as [SHP](https://huggingface.co/datasets/stanfordnlp/SHP) to create this dataset).

| Subset(s) | Factor |
| -------- | ------- |
| politics | Ideologies |
| gender_and_sexuality | Demographics |
| finance, history | Community Norms |
| science | Level of expertise / Community Norms |

### Loading

```python
from datasets import load_dataset

# load finance train set
finance_train_pref = load_dataset("allenai/compred", "finance", split="train_pref")

# load finance test prompts
finance_test_prompts = load_dataset("allenai/compred", "finance_test_prompts", split="test_prompts")
```

### Dataset Structure

Coming soon

<!-- Preference data in each subset (train_pref, validation_pref, test_pref) has the following fields:
- id (str): a unique identifier
- prompt (`str`): the instruction/query which can be safely complied with
- chosen (`str`): the compliant response from a stronger model
- chosen_model (`str`): gpt-4
- rejected (`str`): the noncompliant response from a weaker model
- rejected_model (`str`): where applicable

Test prompts in each subset (test_prompts) has the following fields:

- id (str): a unique identifier
- prompt (str): the instruction/query which should NOT be complied with (original set) or should be complied with (contrast)
- response (str): the noncompliant or compliant response (only in train split)
- category (str): a high-level noncompliance category defined in our taxonomy including: "incomplete requests", "unsupported requests", "indeterminate requests", "humanizing requests", and "requests with safety concerns"
- subcategory (str): a fine-grained subcategory under each category
-->

### Data Creation

Please refer to our [paper](#) for details on our dataset collection.

## Licensing Information

ComPreD is made available under the ODC-BY requiring the user to follow the licenses of the subsequent parts.

## Citation

```
@article{kumar-park2024,
title={{Personalized LMs: Aligning Language Models with Diverse Human Preferences}},
author={Sachin Kumar, Chan Young Park, Yulia Tsvetkov, Noah A. Smith, Hannaneh Hajishirzi},
journal={},
year={2024}
}
```

# 数据集卡片：「ComPRed」

## 数据集详情

### 数据集描述

传统的基于人类反馈训练语言模型（Language Models，LMs）的算法，依赖于被假定适配普通用户的偏好，却忽略了主观性与细粒度的差异。近期有研究指出，将这类多样且往往相互矛盾的人类反馈聚合起来训练单一通用奖励模型（Reward Model）的做法存在争议，人们质疑此类模型最终会对齐何种价值观与用户声音。基于此类奖励最大化目标微调得到的模型，会生成不为众多用户群体所偏好的输出，因为这类模型往往会抹平不同风格与规范的差异，最终沦为通用化模型。为研究这一问题，我们收集并发布了**ComPRed**——一个源自Reddit、带有社区级偏好的问答数据集。该数据集可用于研究偏好多样性，同时不会引发与个体反馈相关的隐私问题。

- **整理方**：艾伦人工智能研究所（Allen Institute for AI）、俄亥俄州立大学、卡内基·梅隆大学
- **论文**：[ArXiv](#)
- **代码仓库**：[https://github.com/allenai/compred](https://github.com/allenai/compred)
- **语言（自然语言处理）**：英语
- **授权协议**：https://opendatacommons.org/licenses/by/odc_by_1.0_public_text.txt
- **联系方式**：[Sachin Kumar与Chan Young Park](mailto:[e-mail], [e-mail])

## 数据集用途

ComPRed包含五个子集，划分依据为引发用户偏好分歧的因素（本数据集的构建流程参考了[SHP](https://huggingface.co/datasets/stanfordnlp/SHP)）。

| 子集名称 | 影响因素 |
| -------- | ------- |
| politics | 意识形态 |
| gender_and_sexuality | 人口统计学特征 |
| finance, history | 社区规范 |
| science | 专业知识水平 / 社区规范 |

### 加载方式

python
from datasets import load_dataset

# 加载金融赛道训练集
finance_train_pref = load_dataset("allenai/compred", "finance", split="train_pref")

# 加载金融赛道测试提示词
finance_test_prompts = load_dataset("allenai/compred", "finance_test_prompts", split="test_prompts")

### 数据集结构

即将公布

### 数据构建

关于数据集收集的详细细节，请参阅我们的[论文](#)。

## 授权信息

ComPRed采用ODC-BY协议发布，使用者需遵守后续相关组件的授权协议。

## 引用格式

bibtex
@article{kumar-park2024,
title={{Personalized LMs: Aligning Language Models with Diverse Human Preferences}},
author={Sachin Kumar, Chan Young Park, Yulia Tsvetkov, Noah A. Smith, Hannaneh Hajishirzi},
journal={},
year={2024}
}