mlmPenguin/pixmo-points

---
license: odc-by
dataset_info:
features:
- name: image_url
dtype: string
- name: image_sha256
dtype: string
- name: points
list:
- name: x
dtype: float64
- name: y
dtype: float64
- name: count
dtype: int64
- name: label
dtype: string
- name: collection_method
dtype: string
splits:
- name: train
num_bytes: 668565775
num_examples: 2376222
download_size: 198336771
dataset_size: 668565775
configs:
- config_name: default
data_files:
- split: train
path: data/train-*
---

# PixMo-Points
PixMo-Points is a dataset of images paired with referring expressions and points marking the locations the
referring expression refers to in the image. It was collected using human annotators and contains a diverse
range of points and expressions, with many high-frequency (10+) expressions.

PixMo-Points is a part of the [PixMo dataset collection](https://huggingface.co/collections/allenai/pixmo-674746ea613028006285687b) and was used to
provide the pointing capabilities of the [Molmo family of models](https://huggingface.co/collections/allenai/molmo-66f379e6fe3b8ef090a8ca19)

Quick links:
- 📃 [Paper](https://molmo.allenai.org/paper.pdf)
- 🎥 [Blog with Videos](https://molmo.allenai.org/blog)

## Loading
```python
data = datasets.load_dataset("allenai/pixmo-points", split="train")
```

## Data Format
Images are stored as URLs that will need to be downloaded separately. Note URLs can be repeated in the data.

The `points` field contains the x, y coordinates specified in pixels.

The `label` field contains the string name of what is being pointed at, this can be a simple object name or a more complex referring expression.

The `collection_method` field specifies whether the image was chosen to target high-frequency counting ("counting") or general pointing ("pointing").

## Image Checking
Image hashes are included to support double-checking that the downloaded image matches the annotated image.
It can be checked like this:

```python
from hashlib import sha256
import requests

example = data[0]
image_bytes = requests.get(example["image_url"]).content
byte_hash = sha256(image_bytes).hexdigest()
assert byte_hash == example["image_sha256"]
```

## License
This dataset is licensed under ODC-BY-1.0. It is intended for research and educational use in accordance with Ai2's [Responsible Use Guidelines](https://allenai.org/responsible-use).

许可证：ODC-BY
数据集信息：
- 字段：image_url，数据类型：字符串
- 字段：image_sha256，数据类型：字符串
- 字段：points，为列表类型，包含两个子字段：
- x：64位浮点数
- y：64位浮点数
- 字段：count，数据类型：整数
- 字段：label，数据类型：字符串
- 字段：collection_method，数据类型：字符串
分割集：
- 分割集名称：train（训练集），占用字节数：668565775，样本数：2376222
下载大小：198336771
数据集总大小：668565775
配置项：
- 配置名称：default（默认配置），数据文件：
- 对应分割集：train（训练集），文件路径：data/train-*

# PixMo-Points 数据集
PixMo-Points 是一类将图像与指代表达式（referring expression）配对的数据集，其中包含用于标记图像中该指代表达式所指向位置的点集。本数据集通过人工标注员采集，涵盖多样的点集与指代表达式，其中包含大量出现频次≥10的高频表达式。

PixMo-Points 属于 [PixMo 数据集合集](https://huggingface.co/collections/allenai/pixmo-674746ea613028006285687b) 的一部分，曾用于赋能 [Molmo 系列模型](https://huggingface.co/collections/allenai/molmo-66f379e6fe3b8ef090a8ca19) 的指向定位能力。

## 快速链接
- 📃 [研究论文](https://molmo.allenai.org/paper.pdf)
- 🎥 [带演示视频的博客](https://molmo.allenai.org/blog)

## 数据集加载
python
data = datasets.load_dataset("allenai/pixmo-points", split="train")

## 数据格式
图像以链接形式存储，需单独下载。请注意数据中可能存在重复的链接。

`points` 字段包含以像素为单位的x、y坐标。

`label` 字段存储被指向目标的字符串名称，既可以是简单的物体名称，也可以是更复杂的指代表达式。

`collection_method` 字段用于指定图像的采集目标：要么针对高频计数任务（取值为`"counting"`），要么用于通用指向任务（取值为`"pointing"`）。

## 图像校验
数据中包含图像哈希值，用于验证下载的图像与标注图像一致，校验代码如下：
python
from hashlib import sha256
import requests

example = data[0]
image_bytes = requests.get(example["image_url"]).content
byte_hash = sha256(image_bytes).hexdigest()
assert byte_hash == example["image_sha256"]

## 许可证
本数据集采用 ODC-BY-1.0 许可证发布，仅可用于研究与教育用途，并需遵循 Ai2 的[负责任使用指南](https://allenai.org/responsible-use)。