describe-anything-dataset

# Dataset Card for Describe Anything Datasets

Datasets used in the training of describe anything models (DAM).

The datasets are in `tar` files. These tar files can be loaded as a webdataset. Alternatively, you can decompress the tar files and use the json file to load the images without using webdatasets.

## Included Datasets

This dataset collection includes annotations and images from the following datasets:

- **COCOStuff** ([COCO-Stuff](http://calvin.inf.ed.ac.uk/datasets/coco-stuff))
- **LVIS** ([LVIS](https://www.lvisdataset.org/))
- **Mapillary** ([Mapillary Vistas 2.0](https://www.mapillary.com/dataset/vistas))
- **OpenImages** ([Open Images V7](https://g.co/dataset/open-images))
- **PACO** ([PACO](https://github.com/facebookresearch/paco))
- **SAM** ([SAM](https://ai.meta.com/datasets/segment-anything-downloads/))
- **SAV** ([SA-V](https://ai.meta.com/datasets/segment-anything-video/))

Each dataset provides localized descriptions used in the training of Describe Anything Models (DAM).

## LICENSE
[NVIDIA Noncommercial License](https://huggingface.co/datasets/nvidia/describe-anything-dataset/blob/main/LICENSE)

## Intended Usage
This dataset is intended to demonstrate and facilitate the understanding and usage of the describe anything models. It should primarily be used for research purposes.

## Ethical Considerations
NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their internal model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse.

Please report security vulnerabilities or NVIDIA AI Concerns [here](https://www.nvidia.com/en-us/support/submit-security-vulnerability/).

# 万物描述（Describe Anything）：精细化本地化图文与视频字幕生成

**英伟达（NVIDIA）、加州大学伯克利分校（UC Berkeley）、加州大学旧金山分校（UCSF）**

[龙联（Long Lian）](https://tonylian.com), [丁一帆（Yifan Ding）](https://research.nvidia.com/person/yifan-ding), [葛云浩（Yunhao Ge）](https://gyhandy.github.io/), [刘思飞（Sifei Liu）](https://sifeiliu.net/), [毛瀚梓（Hanzi Mao）](https://hanzimao.me/), [李博逸（Boyi Li）](https://sites.google.com/site/boyilics/home), [马可·帕沃内（Marco Pavone）](https://research.nvidia.com/person/marco-pavone), [刘明宇（Ming-Yu Liu）](https://mingyuliu.net/), [特雷弗·达雷尔（Trevor Darrell）](https://people.eecs.berkeley.edu/~trevor/), [亚当·亚拉（Adam Yala）](https://www.adamyala.org/), [崔胤（Yin Cui）](https://ycui.me/)

[[论文](https://arxiv.org/abs/2504.16072)] | [[代码](https://github.com/NVlabs/describe-anything)] | [[项目主页](https://describe-anything.github.io/)] | [[演示视频](https://describe-anything.github.io/#video)] | [[HuggingFace 演示Demo](https://huggingface.co/spaces/nvidia/describe-anything-model-demo)] | [[模型/基准测试与数据集](https://huggingface.co/collections/nvidia/describe-anything-680825bb8f5e41ff0785834c)] | [[引用格式](#citation)]

# 万物描述（Describe Anything）数据集卡片

本数据集集合用于训练万物描述模型（Describe Anything Models，简称DAM）。

数据集以`tar`格式打包存储。这些tar文件可作为WebDataset（webdataset）加载；亦可解压tar文件后通过JSON文件直接加载图像数据，无需使用WebDataset。

## 包含的数据集

本数据集集合包含以下数据集的标注与图像数据：
- **COCOStuff**（[COCO-Stuff 数据集](http://calvin.inf.ed.ac.uk/datasets/coco-stuff)）
- **LVIS**（[LVIS 数据集](https://www.lvisdataset.org/)）
- **Mapillary**（[Mapillary Vistas 2.0 数据集](https://www.mapillary.com/dataset/vistas)）
- **OpenImages**（[Open Images V7 数据集](https://g.co/dataset/open-images)）
- **PACO**（[PACO 数据集](https://github.com/facebookresearch/paco)）
- **SAM**（[SAM 数据集](https://ai.meta.com/datasets/segment-anything-downloads/)）
- **SAV**（[SA-V 数据集](https://ai.meta.com/datasets/segment-anything-video/)）

每个数据集均提供本地化描述标注，用于训练万物描述模型（DAM）。

## 许可证
[英伟达（NVIDIA）非商业许可证](https://huggingface.co/datasets/nvidia/describe-anything-dataset/blob/main/LICENSE)

## 预期用途
本数据集旨在演示并助力对万物描述模型的理解与应用，主要用于学术研究场景。

## 伦理考量
英伟达（NVIDIA）认为，可信人工智能是一项共同责任，我们已建立相关政策与实践规范，以支持各类人工智能应用的开发。开发者在按照服务条款下载或使用本模型时，应与内部模型团队协作，确保该模型符合相关行业与应用场景的要求，并防范可能出现的产品误用问题。

请通过[此链接](https://www.nvidia.com/en-us/support/submit-security-vulnerability/)报告安全漏洞或英伟达人工智能相关问题。