pixmo-count

# PixMo-Count
PixMo-Count is a dataset of images paired with objects and their point locations in the image.
It was built by running the [Detic](https://arxiv.org/abs/2201.02605) object detector on web images, and then filtering the data
to improve accuracy and diversity. The val and test sets are human-verified and only contain counts from 2 to 10.

PixMo-Count is a part of the [PixMo dataset collection](https://huggingface.co/collections/allenai/pixmo-674746ea613028006285687b) and was used to
augment the pointing capabilities of the [Molmo family of models](https://huggingface.co/collections/allenai/molmo-66f379e6fe3b8ef090a8ca19)

Quick links:
- 📃 [Paper](https://molmo.allenai.org/paper.pdf)
- 🎥 [Blog with Videos](https://molmo.allenai.org/blog)

## Loading
```python
data = datasets.load_dataset("allenai/pixmo-count", split="train")
```

## Data Format
Images are stored as URLs that will need to be downloaded separately. Note image URLs can be repeated in the data.

The `points` field contains the point x/y coordinates specified in pixels. Missing for the eval sets.

The `label` field contains the string name of the object being pointed at.

The `count` field contains the total count.

## Image Checking
Image hashes are included to support double-checking that the downloaded image matches the annotated image.
It can be checked like this:

```python
from hashlib import sha256
import requests

example = data[0]
image_bytes = requests.get(example["image_url"]).content
byte_hash = sha256(image_bytes).hexdigest()
assert byte_hash == example["image_sha256"]
```

## Splits
The test and val splits are human-verified but do not contain point information. We use them to evaluate counting capabilities of the Molmo models.

## License
This dataset is licensed under ODC-BY-1.0. It is intended for research and educational use in accordance with Ai2's [Responsible Use Guidelines](https://allenai.org/responsible-use).

# PixMo-Count

PixMo-Count 是一类将图像与其中的物体及其点位置进行配对的数据集。该数据集通过在网络图像上运行[Detic](https://arxiv.org/abs/2201.02605)目标检测器构建，随后对数据进行筛选以提升准确率与多样性。其验证集与测试集均经过人工校验，仅包含2至10的计数结果。

PixMo-Count 是[PixMo数据集合集](https://huggingface.co/collections/allenai/pixmo-674746ea613028006285687b)的组成部分，被用于增强[Molmo系列模型](https://huggingface.co/collections/allenai/molmo-66f379e6fe3b8ef090a8ca19)的指向定位能力。

快速链接：
- 📃 [论文](https://molmo.allenai.org/paper.pdf)
- 🎥 [带视频的博客](https://molmo.allenai.org/blog)

## 加载方式
python
data = datasets.load_dataset("allenai/pixmo-count", split="train")

## 数据格式
图像以URL形式存储，需单独下载。需注意，数据中可能存在重复的图像URL。

`points` 字段包含以像素为单位的点x/y坐标，评估集无此字段。

`label` 字段为被指向物体的字符串名称。

`count` 字段为总计数结果。

## 图像校验
数据中包含图像哈希值，用于确认下载的图像与标注图像一致。校验代码如下：
python
from hashlib import sha256
import requests

example = data[0]
image_bytes = requests.get(example["image_url"]).content
byte_hash = sha256(image_bytes).hexdigest()
assert byte_hash == example["image_sha256"]

## 数据集划分
测试集与验证集均经过人工校验，但不包含点坐标信息，我们使用它们来评估Molmo模型的计数能力。

## 许可协议
本数据集采用ODC-BY-1.0许可协议发布，仅可按照Ai2的[负责任使用指南](https://allenai.org/responsible-use)的要求，用于研究与教育用途。