THUDM/ImageRewardDB
收藏资源简介：
--- license: apache-2.0 task_categories: - text-to-image language: - en pretty_name: ImageReward Dataset size_categories: - 100K<n<1M --- # ImageRewardDB ## Dataset Description - **Homepage: https://huggingface.co/datasets/wuyuchen/ImageRewardDB** - **Repository: https://github.com/THUDM/ImageReward** - **Paper: https://arxiv.org/abs/2304.05977** ### Dataset Summary ImageRewardDB is a comprehensive text-to-image comparison dataset, focusing on text-to-image human preference. It consists of 137k pairs of expert comparisons, based on text prompts and corresponding model outputs from DiffusionDB. To build the ImageRewadDB, we design a pipeline tailored for it, establishing criteria for quantitative assessment and annotator training, optimizing labeling experience, and ensuring quality validation. And ImageRewardDB is now publicly available at [🤗 Hugging Face Dataset](https://huggingface.co/datasets/wuyuchen/ImageRewardDB). Notice: All images in ImageRewardDB are collected from DiffusionDB, and in addition, we gathered together images corresponding to the same prompt. ### Languages The text in the dataset is all in English. ### Four Subsets Considering that the ImageRewardDB contains a large number of images, we provide four subsets in different scales to support different needs. For all subsets, the validation and test splits remain the same. The validation split(1.10GB) contains 412 prompts and 2.6K images(7.32K pairs) and the test(1.16GB) split contains 466 prompts and 2.7K images(7.23K pairs). The information on the train split in different scales is as follows: |Subset|Num of Pairs|Num of Images|Num of Prompts|Size| |:--|--:|--:|--:|--:| |ImageRewardDB 1K|17.6K|6.2K|1K|2.7GB| |ImageRewardDB 2K|35.5K|12.5K|2K|5.5GB| |ImageRewardDB 4K|71.0K|25.1K|4K|10.8GB| |ImageRewardDB 8K|141.1K|49.9K|8K|20.9GB| ## Dataset Structure All the data in this repository is stored in a well-organized way. The 62.6K images in ImageRewardDB are split into several folders, stored in corresponding directories under "./images" according to its split. Each folder contains around 500 prompts, their corresponding images, and a JSON file. The JSON file links the image with its corresponding prompt and annotation. The file structure is as follows: ``` # ImageRewardDB ./ ├── images │ ├── train │ │ ├── train_1 │ │ │ ├── 0a1ed3a5-04f6-4a1b-aee6-d584e7c8ed9c.webp │ │ │ ├── 0a58cfa8-ff61-4d31-9757-27322aec3aaf.webp │ │ │ ├── [...] │ │ │ └── train_1.json │ │ ├── train_2 │ │ ├── train_3 │ │ ├── [...] │ │ └── train_32 │ ├── validation │ │ └── [...] │ └── test │ └── [...] ├── metadata-train.parquet ├── metadata-validation.parquet └── metadata-test.parquet ``` The sub-folders have the name of {split_name}_{part_id}, and the JSON file has the same name as the sub-folder. Each image is a lossless WebP file and has a unique name generated by [UUID](https://en.wikipedia.org/wiki/Universally_unique_identifier). ### Data Instances For instance, below is the image of `1b4b2d61-89c2-4091-a1c0-f547ad5065cb.webp` and its information in train_1.json. ```json { "image_path": "images/train/train_1/0280642d-f69f-41d1-8598-5a44e296aa8b.webp", "prompt_id": "000864-0061", "prompt": "painting of a holy woman, decorated, intricate, elegant, highly detailed, digital painting, artstation, concept art, smooth, sharp focus, illustration, art by artgerm and greg rutkowski and alphonse mucha, 8 k ", "classification": "People", "image_amount_in_total": 9, "rank": 5, "overall_rating": 4, "image_text_alignment_rating": 3, "fidelity_rating": 4 } ``` ### Data Fields * image: The image object * prompt_id: The id of the corresponding prompt * prompt: The text of the corresponding prompt * classification: The classification of the corresponding prompt * image_amount_in_total: Total amount of images related to the prompt * rank: The relative rank of the image in all related images * overall_rating: The overall score of this image * image_text_alignment_rating: The score of how well the generated image matches the given text * fidelity_rating: The score of whether the output image is true to the shape and characteristics that the object should have ### Data Splits As we mentioned above, all scales of the subsets we provided have three splits of "train", "validation", and "test". And all the subsets share the same validation and test splits. ### Dataset Metadata We also include three metadata tables `metadata-train.parquet`, `metadata-validation.parquet`, and `metadata-test.parquet` to help you access and comprehend ImageRewardDB without downloading the Zip files. All the tables share the same schema, and each row refers to an image. The schema is shown below, and actually, the JSON files we mentioned above share the same schema: |Column|Type|Description| |:---|:---|:---| |`image_path`|`string`|The relative path of the image in the repository.| |`prompt_id`|`string`|The id of the corresponding prompt.| |`prompt`|`string`|The text of the corresponding prompt.| |`classification`|`string`| The classification of the corresponding prompt.| |`image_amount_in_total`|`int`| Total amount of images related to the prompt.| |`rank`|`int`| The relative rank of the image in all related images.| |`overall_rating`|`int`| The overall score of this image. |`image_text_alignment_rating`|`int`|The score of how well the generated image matches the given text.| |`fidelity_rating`|`int`|The score of whether the output image is true to the shape and characteristics that the object should have.| Below is an example row from metadata-train.parquet. |image_path|prompt_id|prompt|classification|image_amount_in_total|rank|overall_rating|image_text_alignment_rating|fidelity_rating| |:---|:---|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---|:---|:---|:---|:---|:---| |images/train/train_1/1b4b2d61-89c2-4091-a1c0-f547ad5065cb.webp|001324-0093|a magical forest that separates the good world from the dark world, ...|Outdoor Scenes|8|3|6|6|6| ## Loading ImageRewardDB You can use the Hugging Face [Datasets](https://huggingface.co/docs/datasets/quickstart) library to easily load the ImageRewardDB. As we mentioned before, we provide four subsets in the scales of 1k, 2k, 4k, and 8k. You can load them using as following: ```python from datasets import load_dataset # Load the 1K-scale dataset dataset = load_dataset("THUDM/ImageRewardDB", "1k") # Load the 2K-scale dataset dataset = load_dataset("THUDM/ImageRewardDB", "2k") # Load the 4K-scale dataset dataset = load_dataset("THUDM/ImageRewardDB", "4K") # Load the 8K-scale dataset dataset = load_dataset("THUDM/ImageRewardDB", "8k") ``` ## Additional Information ### Licensing Information The ImageRewardDB dataset is available under the [Apache license 2.0](https://www.apache.org/licenses/LICENSE-2.0.html). The Python code in this repository is available under the [MIT License](https://github.com/poloclub/diffusiondb/blob/main/LICENSE). ### Citation Information ``` @misc{xu2023imagereward, title={ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generation}, author={Jiazheng Xu and Xiao Liu and Yuchen Wu and Yuxuan Tong and Qinkai Li and Ming Ding and Jie Tang and Yuxiao Dong}, year={2023}, eprint={2304.05977}, archivePrefix={arXiv}, primaryClass={cs.CV} } ```
--- 许可证: Apache 2.0 任务类别: - 文本到图像生成 语言: - 英语 数据集名称: ImageReward数据集 样本量区间: - 100K<n<1M --- # ImageRewardDB ## 数据集描述 - **主页：https://huggingface.co/datasets/wuyuchen/ImageRewardDB** - **代码仓库：https://github.com/THUDM/ImageReward** - **论文：https://arxiv.org/abs/2304.05977** ### 数据集概览 ImageRewardDB是一款面向文本到图像人类偏好的综合性文本到图像对比数据集，共计包含13.7万对专家标注对比样本，其数据来源于DiffusionDB（扩散数据库）的文本提示词与对应模型生成图像。为构建该数据集，我们设计了定制化处理流程，建立了量化评估标准与标注员培训机制，优化了标注体验并确保了质量校验。目前ImageRewardDB已在[🤗 Hugging Face数据集平台](https://huggingface.co/datasets/wuyuchen/ImageRewardDB)公开。 注意：ImageRewardDB内的所有图像均采集自DiffusionDB，此外我们还收集了同一提示词对应的所有关联图像。 ### 语言 数据集内所有文本均为英语。 ### 四个子集 考虑到ImageRewardDB包含海量图像，我们提供了四个不同规模的子集以适配各类需求。所有子集的验证集与测试集划分保持一致。其中验证集（1.10GB）包含412个提示词与2.6万张图像（对应7.32万对样本），测试集（1.16GB）包含466个提示词与2.7万张图像（对应7.23万对样本）。不同规模训练集的详细信息如下： |子集|样本对数量|图像总数|提示词数量|体积| |:--|--:|--:|--:|--:| |ImageRewardDB 1K|17.6K|6.2K|1K|2.7GB| |ImageRewardDB 2K|35.5K|12.5K|2K|5.5GB| |ImageRewardDB 4K|71.0K|25.1K|4K|10.8GB| |ImageRewardDB 8K|141.1K|49.9K|8K|20.9GB| ## 数据集结构 本仓库内所有数据均经过有序组织。ImageRewardDB中的6.26万张图像被拆分至多个文件夹，按照数据集划分存储在`./images`目录下的对应子目录中。每个文件夹包含约500个提示词及其对应的图像与一个JSON文件，该JSON文件用于关联图像、提示词与标注信息。文件结构如下： # ImageRewardDB ./ ├── images │ ├── train │ │ ├── train_1 │ │ │ ├── 0a1ed3a5-04f6-4a1b-aee6-d584e7c8ed9c.webp │ │ │ ├── 0a58cfa8-ff61-4d31-9757-27322aec3aaf.webp │ │ │ ├── [...] │ │ │ └── train_1.json │ │ ├── train_2 │ │ ├── train_3 │ │ ├── [...] │ │ └── train_32 │ ├── validation │ │ └── [...] │ └── test │ └── [...] ├── metadata-train.parquet ├── metadata-validation.parquet └── metadata-test.parquet 子文件夹的命名格式为`{split_name}_{part_id}`，JSON文件与对应子文件夹同名。每张图像均为无损WebP（WebP）格式文件，文件名采用通用唯一识别码（Universally Unique Identifier, UUID）生成，具有全局唯一性。 ### 数据样例 以`train_1.json`中的`1b4b2d61-89c2-4091-a1c0-f547ad5065cb.webp`图像及其标注信息为例： json { "image_path": "images/train/train_1/0280642d-f69f-41d1-8598-5a44e296aa8b.webp", "prompt_id": "000864-0061", "prompt": "painting of a holy woman, decorated, intricate, elegant, highly detailed, digital painting, artstation, concept art, smooth, sharp focus, illustration, art by artgerm and greg rutkowski and alphonse mucha, 8 k ", "classification": "People", "image_amount_in_total": 9, "rank": 5, "overall_rating": 4, "image_text_alignment_rating": 3, "fidelity_rating": 4 } ### 数据字段 * `image`：图像对象 * `prompt_id`：对应提示词的唯一标识 * `prompt`：对应提示词的文本内容 * `classification`：对应提示词的分类标签 * `image_amount_in_total`：该提示词对应的总图像数量 * `rank`：该图像在所有关联图像中的相对排名 * `overall_rating`：该图像的综合评分 * `image_text_alignment_rating`：生成图像与给定文本的匹配程度评分 * `fidelity_rating`：生成图像是否符合对象固有形状与特征的保真度评分 ### 数据划分 如前文所述，我们提供的所有规模子集均包含“训练集”“验证集”“测试集”三个划分，且所有子集共享同一套验证集与测试集。 ### 数据集元数据 我们还提供了三个元数据表`metadata-train.parquet`、`metadata-validation.parquet`与`metadata-test.parquet`，方便用户无需下载完整压缩包即可访问并理解ImageRewardDB。 所有表的 schema 均保持一致，每一行对应一张图像，其 schema 如下（与前文提及的JSON文件 schema 完全一致）： |列名|数据类型|描述| |:---|:---|:---| |`image_path`|`string`|图像在仓库中的相对路径| |`prompt_id`|`string`|对应提示词的唯一标识| |`prompt`|`string`|对应提示词的文本内容| |`classification`|`string`|对应提示词的分类标签| |`image_amount_in_total`|`int`|该提示词对应的总图像数量| |`rank`|`int`|该图像在所有关联图像中的相对排名| |`overall_rating`|`int`|该图像的综合评分| |`image_text_alignment_rating`|`int`|生成图像与给定文本的匹配程度评分| |`fidelity_rating`|`int`|生成图像是否符合对象固有形状与特征的保真度评分| 以下是`metadata-train.parquet`中的一行样例数据： |image_path|prompt_id|prompt|classification|image_amount_in_total|rank|overall_rating|image_text_alignment_rating|fidelity_rating| |:---|:---|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---|:---|:---|:---|:---|:---| |images/train/train_1/1b4b2d61-89c2-4091-a1c0-f547ad5065cb.webp|001324-0093|a magical forest that separates the good world from the dark world, ...|Outdoor Scenes|8|3|6|6|6| ## 加载ImageRewardDB 您可以使用Hugging Face [Datasets](https://huggingface.co/docs/datasets/quickstart)库轻松加载ImageRewardDB。如前文所述，我们提供了1k、2k、4k、8k四种规模的子集，加载代码如下： python from datasets import load_dataset # 加载1K规模数据集 dataset = load_dataset("THUDM/ImageRewardDB", "1k") # 加载2K规模数据集 dataset = load_dataset("THUDM/ImageRewardDB", "2k") # 加载4K规模数据集 dataset = load_dataset("THUDM/ImageRewardDB", "4K") # 加载8K规模数据集 dataset = load_dataset("THUDM/ImageRewardDB", "8k") ## 补充信息 ### 授权信息 ImageRewardDB数据集采用[Apache许可证2.0](https://www.apache.org/licenses/LICENSE-2.0.html)进行授权。本仓库内的Python代码采用[MIT许可证](https://github.com/poloclub/diffusiondb/blob/main/LICENSE)进行授权。 ### 引用信息 @misc{xu2023imagereward, title={ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generation}, author={Jiazheng Xu and Xiao Liu and Yuchen Wu and Yuxuan Tong and Qinkai Li and Ming Ding and Jie Tang and Yuxiao Dong}, year={2023}, eprint={2304.05977}, archivePrefix={arXiv}, primaryClass={cs.CV} }
数据集概述
数据集名称: ImageRewardDB
数据集描述: ImageRewardDB 是一个专注于文本到图像人类偏好的综合比较数据集。它包含137k对专家比较，基于文本提示和来自DiffusionDB的相应模型输出。数据集通过专门设计的管道进行构建，包括量化评估标准、注释者培训、标签体验优化和质量验证。
语言: 数据集中的文本均为英语。
数据集结构: 数据集中的62.6K图像根据其分割存储在"./images"下的相应目录中。每个目录包含约500个提示、相应的图像和一个JSON文件，该文件将图像与其相应的提示和注释链接。
数据集大小: 数据集包含四个不同规模的子集，每个子集的训练、验证和测试分割大小如下：
| 子集 | 对数 | 图像数 | 提示数 | 大小 |
|---|---|---|---|---|
| ImageRewardDB 1K | 17.6K | 6.2K | 1K | 2.7GB |
| ImageRewardDB 2K | 35.5K | 12.5K | 2K | 5.5GB |
| ImageRewardDB 4K | 71.0K | 25.1K | 4K | 10.8GB |
| ImageRewardDB 8K | 141.1K | 49.9K | 8K | 20.9GB |
数据字段:
- image: 图像对象
- prompt_id: 对应提示的ID
- prompt: 对应提示的文本
- classification: 对应提示的分类
- image_amount_in_total: 与提示相关的图像总数
- rank: 图像在所有相关图像中的相对排名
- overall_rating: 图像的整体评分
- image_text_alignment_rating: 生成图像与给定文本匹配程度的评分
- fidelity_rating: 输出图像是否忠实于对象应有的形状和特征的评分
数据分割: 所有规模的子集都包含"train", "validation", 和 "test"三个分割，且所有子集共享相同的验证和测试分割。
许可证: 数据集根据Apache许可证2.0提供。
