HaoyeZhang/RLHF-V-Dataset

---
license: cc-by-nc-4.0
task_categories:
- text-generation
- visual-question-answering
language:
- en
configs:
- config_name: default
data_files: RLHF-V-Dataset.parquet
dataset_info:
features:
- name: ds_name
dtype: string
- name: image
dtype: image
- name: text
dtype: string
- name: origin_dataset
dtype: string
- name: origin_split
dtype: string
- name: idx
dtype: int64
- name: image_path
dtype: string
pretty_name: RLHF-V-Dataset
size_categories:
- 1K<n<10K
---
# Dataset Card for RLHF-V-Dataset
[Project Page](https://rlhf-v.github.io/) | [Paper](https://arxiv.org/abs/2312.00849) | [GitHub](https://github.com/RLHF-V/RLHF-V)

## Updates
**[2024.01.06]** 🔥 **A larger, more diverse set of fine-grained human correction data is available now!** 🔥 The newly released data has about **5.7k of fine-grained human correction data** that covers the output of **more powerful models** (Qwen-VL-Chat, InstructBLIP, etc.). We also **expand the image types** from everyday scenes to diverse styles and themes (WikiArt, landmarks, scene texts, etc.).

**[2024.01.05]** 🔧 We reformat our dataset and now it is **more convenient to preview and use** our data! The dataset now supports the `load_dataset` function, and the data content can be easily previewed online.

**[2023.12.15]** We incorporated a new annotation subset with an additional **1065 fine-grained annotations** into our dataset !

## Dataset Summary
RLHF-V-Dataset is the human preference data used in "**RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedback**".

We originally collected a large amount of **fine-grained segment-level human corrections** on diverse instructions, including detailed descriptions and question-answering instructions. More high-quality annotations for different image sources and model outputs are on the way.

<p align="center">
<img src="https://cdn-uploads.huggingface.co/production/uploads/6566e0c493e30c8a60048eb3/jerEZiHDDc2ceF9anVHR-.png" alt="fig1" width="60%"/>
</p>

Utilizing our dataset can dramatically **reduce model hallucinations by 34.8%** while **keeping informativeness**.

<p align="center">
<img src="https://cdn-uploads.huggingface.co/production/uploads/6566e0c493e30c8a60048eb3/7xJEdKXeW33iKdHqJwvNN.png" alt="fig2" width="70%"/>
</p>

## Usage
```python
from datasets import load_dataset

data = load_dataset("HaoyeZhang/RLHF-V-Dataset")
```

## Data fields

| | Key | Description |
| ---- | ---------------- | ------------------------------------------------------------ |
| 0 | `ds_name` | Dataset name. |
| 1 | `image` | Dict contains path and bytes. If loaded by `load_dataset`, it can be automatically converted into a PIL Image. |
| 2 | `text` | Preference data. Each data item contains a dict with the keys "question", "chosen", and "rejected". |
| 3 | `origin_dataset` | Original dataset for annotation, which is not used in training. |
| 4 | `origin_split` | Meta information for each data item, including the name of the model we use to generate the original answer, and the question type ("detailed description" or "question answering") |
| 5 | `idx` | Data index. |
| 6 | `image_path` | Image path. |

## Citation

```
@article{yu2023rlhf,
title={Rlhf-v: Towards trustworthy mllms via behavior alignment from fine-grained correctional human feedback},
author={Yu, Tianyu and Yao, Yuan and Zhang, Haoye and He, Taiwen and Han, Yifeng and Cui, Ganqu and Hu, Jinyi and Liu, Zhiyuan and Zheng, Hai-Tao and Sun, Maosong and others},
journal={arXiv preprint arXiv:2312.00849},
year={2023}
}
```

原始信息汇总

数据集概述

基本信息

• 数据集名称：RLHF-V-Dataset
• 许可证：cc-by-nc-4.0
• 语言：英语
• 配置：默认配置，数据文件为RLHF-V-Dataset.parquet
• 大小范围：1K<n<10K

数据集特征

• ds_name：数据集名称，数据类型为字符串。
• image：图像数据，数据类型为图像。
• text：文本数据，数据类型为字符串。
• origin_dataset：原始数据集，数据类型为字符串。
• origin_split：原始分割信息，数据类型为字符串。
• idx：数据索引，数据类型为int64。
• image_path：图像路径，数据类型为字符串。

数据集用途

• 任务类别：文本生成、视觉问答
• 应用：用于"RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedback"研究，主要用于减少模型幻觉并保持信息性。

数据集更新

• 2024.01.06：新增约5.7k的细粒度人类校正数据，扩展了图像类型。
• 2024.01.05：数据集格式更新，支持load_dataset函数，便于预览和使用。
• 2023.12.15：新增1065个细粒度注释。

数据集加载示例

python from datasets import load_dataset

data = load_dataset("HaoyeZhang/RLHF-V-Dataset")

数据字段描述

• ds_name：数据集名称。
• image：包含路径和字节的字典，可自动转换为PIL图像。
• text：偏好数据，每个数据项包含"question", "chosen", "rejected"键。
• origin_dataset：原始注释数据集。
• origin_split：元信息，包括模型名称和问题类型。
• idx：数据索引。
• image_path：图像路径。