Flux_SD3_MJ_Dalle_Human_Coherence_Dataset

## **NOTE:** A newer version of this dataset is available: [Imagen3_Flux1.1_Flux1_SD3_MJ_Dalle_Human_Coherence_Dataset](https://huggingface.co/datasets/Rapidata/Imagen3_Flux1.1_Flux1_SD3_MJ_Dalle_Human_Coherence_Dataset)

# Rapidata Image Generation Coherence Dataset

<a href="https://www.rapidata.ai">
<img src="https://cdn-uploads.huggingface.co/production/uploads/66f5624c42b853e73e0738eb/jfxR79bOztqaC6_yNNnGU.jpeg" width="400" alt="Dataset visualization">
</a>

This Dataset is a 1/3 of a 2M+ human annotation dataset that was split into three modalities: Preference, Coherence, Text-to-Image Alignment.
- Link to the Preference dataset: https://huggingface.co/datasets/Rapidata/700k_Human_Preference_Dataset_FLUX_SD3_MJ_DALLE3
- Link to the Text-2-Image Alignment dataset: https://huggingface.co/datasets/Rapidata/Flux_SD3_MJ_Dalle_Human_Alignment_Dataset

It was collected using the Rapidata Python API https://docs.rapidata.ai

If you get value from this dataset and would like to see more in the future, please consider liking it.

## Overview
One of the largest human annotated coherence datasets for text-to-image models, this release contains over 700,000 human votes when asked which generated image is more coherent - one third of our complete 2 million vote collection. This preference dataset is part of a larger evaluation comparing images from leading AI models including Flux.1, DALL-E 3, MidJourney, and Stable Diffusion. The complete collection includes two additional datasets of equal size focusing on image preference and text-image alignment, available on our profile. This extensive dataset was collected in just 2 days using Rapidata's groundbreaking annotation technology, demonstrating unprecedented efficiency in large-scale human feedback collection.

Explore our latest model rankings on our [website](https://www.rapidata.ai/benchmark).

## Key Features
- **Massive Scale**: 700,000+ individual human preference votes collected in 48 hours
- **Global Representation**: Collected from 144,292 participants across 145 countries
- **Diverse Prompts**: 282 carefully curated prompts testing various aspects of image generation
- **Leading Models**: Comparisons between four state-of-the-art image generation models
- **Rigorous Methodology**: Uses pairwise comparisons with built-in quality controls
- **Rich Demographic Data**: Includes annotator information about age, gender, and geographic location

<img src="https://cdn-uploads.huggingface.co/production/uploads/66f5624c42b853e73e0738eb/1LVQj_G5bFio7w4WXPxsC.png" alt="Image description" width="650">
Figure: Overview of the distribution of annotators by continent (left) compared to the world population distribution(right)

## Applications
This dataset is invaluable for:
- Training and fine-tuning image generation models
- Understanding global preferences in AI-generated imagery
- Developing better evaluation metrics for generative models
- Researching cross-cultural aesthetic preferences
- Benchmarking new image generation models

## Data Collection Powered by Rapidata
What traditionally would take weeks or months of data collection was accomplished in just 48 hours through Rapidata's innovative annotation platform. Our technology enables:
- Lightning-fast data collection at massive scale
- Global reach across 145+ countries
- Built-in quality assurance mechanisms
- Comprehensive demographic representation
- Cost-effective large-scale annotation

## Citation
If you use this dataset in your research, please cite our Startup Rapidata and our paper: "Finding the Subjective Truth: Collecting 2 Million Votes for Comprehensive Gen-AI Model Evaluation" (arXiv:2409.11904v2)
```
@misc{christodoulou2024findingsubjectivetruthcollecting,
title={Finding the Subjective Truth: Collecting 2 Million Votes for Comprehensive Gen-AI Model Evaluation},
author={Dimitrios Christodoulou and Mads Kuhlmann-Jørgensen},
year={2024},
eprint={2409.11904},
archivePrefix={arXiv},
primaryClass={cs.CV},
url={https://arxiv.org/abs/2409.11904},
}
```

## About Rapidata
Rapidata's technology makes collecting human feedback at scale faster and more accessible than ever before. Visit [rapidata.ai](https://www.rapidata.ai/) to learn more about how we're revolutionizing human feedback collection for AI development.

We created the dataset using our in-house developed [API](https://docs.rapidata.ai/), which you can access to gain near-instant human intelligence.

## 注意：本数据集已有更新版本：[Imagen3_Flux1.1_Flux1_SD3_MJ_Dalle_Human_Coherence_Dataset](https://huggingface.co/datasets/Rapidata/Imagen3_Flux1.1_Flux1_SD3_MJ_Dalle_Human_Coherence_Dataset)

# Rapidata 图像生成一致性数据集

<a href="https://www.rapidata.ai">
<img src="https://cdn-uploads.huggingface.co/production/uploads/66f5624c42b853e73e0738eb/jfxR79bOztqaC6_yNNnGU.jpeg" width="400" alt="数据集可视化">
</a>

本数据集是包含200万+人类标注的完整数据集的三分之一，该完整数据集被划分为三个任务方向：偏好（Preference）、一致性（Coherence）、文本到图像对齐（Text-to-Image Alignment）。
- 偏好数据集链接：https://huggingface.co/datasets/Rapidata/700k_Human_Preference_Dataset_FLUX_SD3_MJ_DALLE3
- 文本到图像对齐数据集链接：https://huggingface.co/datasets/Rapidata/Flux_SD3_MJ_Dalle_Human_Alignment_Dataset

本数据集通过Rapidata Python API（https://docs.rapidata.ai）完成采集。

若您从本数据集获益并希望后续获取更多相关资源，请考虑为该数据集点赞。

## 概览

本数据集是目前规模最大的面向文本到图像生成模型的人类标注一致性数据集之一，本次发布包含超过70万条针对「哪张生成图像更具一致性」的人类投票数据——该数据为我们完整的200万条投票集合的三分之一。本偏好数据集是一项大型评估任务的组成部分，该评估对Flux.1、DALL-E 3、MidJourney以及Stable Diffusion等主流AI模型生成的图像进行对比。完整数据集集合还包含另外两个同等规模的数据集，分别聚焦图像偏好与文本到图像对齐任务，可在我们的Hugging Face主页获取。本大规模数据集仅用2天便完成采集，依托Rapidata突破性的标注技术，展现了大规模人类反馈采集领域前所未有的效率。

您可通过我们的[官方网站](https://www.rapidata.ai/benchmark)查看最新的模型排行榜。

## 核心特性
- **超大规模**：48小时内采集超过70万条独立人类偏好投票
- **全球覆盖**：采集自145个国家的144292名参与者
- **多样化提示词**：包含282条精心筛选的提示词，覆盖图像生成的多维度测试场景
- **主流模型对比**：涵盖四款当前最先进的图像生成模型的对比评估
- **严谨的实验范式**：采用成对比较法，并内置质量控制机制
- **丰富的人口统计数据**：包含标注者的年龄、性别与地理位置信息

<img src="https://cdn-uploads.huggingface.co/production/uploads/66f5624c42b853e73e0738eb/1LVQj_G5bFio7w4WXPxsC.png" alt="图像说明" width="650">
图：按大洲划分的标注者分布概览（左）与全球人口分布（右）对比

## 应用场景

本数据集可广泛应用于：
- 训练与微调图像生成模型
- 探究AI生成图像的全球审美偏好
- 为生成式模型研发更完善的评估指标
- 开展跨文化审美偏好相关研究
- 为新型图像生成模型提供基准测试支持

## 基于Rapidata的数据采集

传统数据采集往往需要数周乃至数月，而依托Rapidata创新性的标注平台，本数据集仅用48小时便完成采集。我们的技术具备以下优势：
- 支持超大规模的极速数据采集
- 覆盖145个以上国家的全球采集能力
- 内置质量保障机制
- 全面的人口统计样本覆盖
- 具备成本效益的大规模标注能力

## 引用方式

若您在研究中使用本数据集，请引用我们的初创团队Rapidata以及论文："Finding the Subjective Truth: Collecting 2 Million Votes for Comprehensive Gen-AI Model Evaluation"（arXiv:2409.11904v2）

@misc{christodoulou2024findingsubjectivetruthcollecting,
title={Finding the Subjective Truth: Collecting 2 Million Votes for Comprehensive Gen-AI Model Evaluation},
author={Dimitrios Christodoulou and Mads Kuhlmann-Jørgensen},
year={2024},
eprint={2409.11904},
archivePrefix={arXiv},
primaryClass={cs.CV},
url={https://arxiv.org/abs/2409.11904},
}

## 关于Rapidata

Rapidata的技术让大规模人类反馈采集比以往任何时候都更加快捷与易用。请访问[rapidata.ai](https://www.rapidata.ai/)了解更多关于我们如何革新AI开发领域的人类反馈采集技术的信息。

我们通过自研的[API](https://docs.rapidata.ai/)构建了本数据集，您可通过该接口获取近乎实时的人类智能标注服务。