VLFeedback

# Dataset Card for VLFeedback

- **Homepage:** https://vlf-silkie.github.io/
- **Repository:** https://github.com/vlf-silkie/VLFeedback
- **Paper:** https://arxiv.org/abs/2312.10665

## Dataset Summary

VLFeedback is a **large-scale vision-language preference dataset**, annotated by GPT-4V. It consists of 80k multi-modal instructions from various souces that encompass various capabilities of LVLMs.

<p align="center">
<img src="https://cdn-uploads.huggingface.co/production/uploads/622f103fc78da4c7ebd7c887/kDCFRInpUoEVLaK-1T1Bp.png" alt="fig1" width="60%"/>
</p>

We build a model pool of 12 LVLMs and each data sample contains 4 responses from different models. Each response is annotated in three aspects: **helpfulness**, **visual faithfulness**, and **ethical considerations**. The resulting preference dataset contains **more than 380k comparison pairs**.

<p align="center">
<img src="https://cdn-uploads.huggingface.co/production/uploads/622f103fc78da4c7ebd7c887/zOLje1p2ytJ27Ml2kJKhI.png" alt="fig2" width="60%"/>
</p>

## Citation

```
@article{2023vlfeedback,
author = {Lei Li and Zhihui Xie and Mukai Li and Shunian Chen and Peiyi Wang and Liang Chen and Yazheng Yang and Benyou Wang and Lingpeng Kong},
title = {Silkie: Preference Distillation for Large Visual Language Models},
publisher = {arXiv:2312.10665},
year = {2023}
}
```

# VLFeedback 数据集卡片
- **项目主页：** https://vlf-silkie.github.io/
- **代码仓库：** https://github.com/vlf-silkie/VLFeedback
- **论文链接：** https://arxiv.org/abs/2312.10665

## 数据集概述
VLFeedback 是一款**大规模视觉语言偏好数据集（large-scale vision-language preference dataset）**，由GPT-4V完成标注。该数据集包含来自多类来源的8万条多模态指令，覆盖了大型视觉语言模型（Large Visual Language Models, LVLMs）的各类能力。

<p align="center">
<img src="https://cdn-uploads.huggingface.co/production/uploads/622f103fc78da4c7ebd7c887/kDCFRInpUoEVLaK-1T1Bp.png" alt="图1" width="60%"/>
</p>

我们构建了包含12个大型视觉语言模型的模型池，每个数据样本均包含来自不同模型的4条回复。每条回复从**有用性（helpfulness）**、**视觉忠实性（visual faithfulness）**和**伦理考量（ethical considerations）**三个维度进行标注。最终生成的偏好数据集包含**超过38万条比较样本对**。

<p align="center">
<img src="https://cdn-uploads.huggingface.co/production/uploads/622f103fc78da4c7ebd7c887/zOLje1p2ytJ27Ml2kJKhI.png" alt="图2" width="60%"/>
</p>

## 引用

@article{2023vlfeedback,
author = {"Lei Li and Zhihui Xie and Mukai Li and Shunian Chen and Peiyi Wang and Liang Chen and Yazheng Yang and Benyou Wang and Lingpeng Kong"},
title = {"Silkie: Preference Distillation for Large Visual Language Models"},
publisher = {"arXiv:2312.10665"},
year = {2023}
}