pixmo-cap-qa

# PixMo-CapQA
PixMo-CapQA is a synthetic dataset of question/answer pairs about images. The data was generated by using the
[Claude](https://www.anthropic.com/claude) large language model to build Q/A pairs from [dense captions of images](https://huggingface.co/datasets/allenai/pixmo-cap) (the model did not see the actual images).

PixMo-CapQA is a part of the [PixMo dataset collection](https://huggingface.co/collections/allenai/pixmo-674746ea613028006285687b) and was used to train the [Molmo family of models](https://huggingface.co/collections/allenai/molmo-66f379e6fe3b8ef090a8ca19)

Quick links:
- 📃 [Paper](https://molmo.allenai.org/paper.pdf)
- 🎥 [Blog with Videos](https://molmo.allenai.org/blog)

## Loading
```python
data = datasets.load_dataset("allenai/pixmo-cap-qa", split="train")
```

## Data Format
Images are stored as URLs that will need to be downloaded separately.
The image URLs can be repeated since many of the images have multiple Q/A pairs.

- The `question` field contains the input text, it includes "[USER]" and "[ASSISTANT]" tags
- The `answer` field contains the final target output text
- The `messages` field contains the same data in a list-of-messages formats. The first message is from the
user, then messages alternative between user and assistant. This text does not contain "[USER]" and "[ASSISTANT]" tags

## License
This dataset is licensed under ODC-BY-1.0. It is intended for research and educational use in accordance with Ai2's [Responsible Use Guidelines](https://allenai.org/responsible-use).
This dataset includes data generated from Claude which are subject to Anthropic [terms of service](https://www.anthropic.com/legal/commercial-terms) and [usage policy](https://www.anthropic.com/legal/aup).

# PixMo-CapQA
PixMo-CapQA是一款面向图像的问答对合成数据集。该数据集通过[Claude](https://www.anthropic.com/claude)大语言模型（Large Language Model），基于图像密集字幕（dense captions）构建问答对，且模型未接触过原始图像。

PixMo-CapQA隶属于[PixMo数据集合集](https://huggingface.co/collections/allenai/pixmo-674746ea613028006285687b)，曾用于训练[Molmo系列模型](https://huggingface.co/collections/allenai/molmo-66f379e6fe3b8ef090a8ca19)。

## 快速链接
- 📃 [论文](https://molmo.allenai.org/paper.pdf)
- 🎥 [含视频的博客](https://molmo.allenai.org/blog)

## 加载方法
python
data = datasets.load_dataset("allenai/pixmo-cap-qa", split="train")

## 数据格式
图像以URL形式存储，需单独下载。由于部分图像对应多组问答对，图像URL可能重复出现。
- `question`字段存储输入文本，其中包含"[USER]"与"[ASSISTANT]"标签
- `answer`字段存储最终目标输出文本
- `messages`字段以消息列表格式存储相同数据：首条消息来自用户，后续消息在用户与助手间交替，且文本中不包含"[USER]"与"[ASSISTANT]"标签

## 许可协议
本数据集采用ODC-BY-1.0许可协议，仅可用于符合AI2[负责任使用指南](https://allenai.org/responsible-use)的研究与教育用途。
本数据集包含由Claude生成的数据，需遵守Anthropic的[服务条款](https://www.anthropic.com/legal/commercial-terms)与[使用政策](https://www.anthropic.com/legal/aup).