Rapidata/Flux-2-pro_t2i_human_preference

---
configs:
- config_name: default
data_files:
- split: train
path: data/train-*
dataset_info:
features:
- name: prompt
dtype: string
- name: image1
dtype: image
- name: image2
dtype: image
- name: model1
dtype: string
- name: model2
dtype: string
- name: weighted_results_image1_preference
dtype: float32
- name: weighted_results_image2_preference
dtype: float32
- name: detailed_results_preference
dtype: string
- name: weighted_results_image1_coherence
dtype: float32
- name: weighted_results_image2_coherence
dtype: float32
- name: detailed_results_coherence
dtype: string
- name: weighted_results_image1_alignment
dtype: float32
- name: weighted_results_image2_alignment
dtype: float32
- name: detailed_results_alignment
dtype: string
splits:
- name: train
num_bytes: 34055275649
num_examples: 44857
download_size: 28906555375
dataset_size: 34055275649
license: cdla-permissive-2.0
task_categories:
- text-to-image
- image-to-text
- image-classification
- reinforcement-learning
language:
- en
tags:
- Human
- Preference
- Coherence
- Alignment
- country
- language
- flux
- midjourney
- dalle3
- stabeldiffusion
- alignment
- flux1.1
- flux1
- imagen3
- aurora
- lumina
- recraft
- recraft v2
- ideogram
- frames
- OpenAI 4o
- 4o
- OpenAI
- Seedream-3
- seedream
- Imagen-4
- Google
- Recraft v3
- Hunyuan Image 2.1
- Flux 2 Pro
size_categories:
- 10K<n<100K
pretty_name: Flux 2 Pro vs Hunyuan Image 2.1 / Recraft v3 / 4 Ultra 24.7.25 / Seedream 3 / Ideogram V2 / Recraft V2 / Lumina-15-2-25 / Frames-23-1-25
/ Aurora / imagen-3 / Flux-1.1-pro / Flux-1-pro / Dalle-3 / Midjourney-5.2 / Stabel-Diffusion-3 / 4o
- Human Preference Dataset
---

<style>

.vertical-container {
display: flex;
flex-direction: column;
gap: 60px;
}

.horizontal-container {
display: flex;
flex-direction: row;
justify-content: center;
gap: 60px;
}

.image-container img {
max-height: 250px; /* Set the desired height */
margin:0;
object-fit: contain; /* Ensures the aspect ratio is maintained */
width: auto; /* Adjust width automatically based on height */
box-sizing: content-box;
}

.image-container img.big {
max-height: 350px; /* Set the desired height */
}

.image-container {
display: flex; /* Aligns images side by side */
justify-content: space-around; /* Space them evenly */
align-items: center; /* Align them vertically */
gap: .5rem
}

.container {
width: 90%;
margin: 0 auto;
}

.text-center {
text-align: center;
}

.score-amount {
margin: 0;
margin-top: 10px;
}

.score-percentage {Score:
font-size: 12px;
font-weight: semi-bold;
}

.link-container {
padding: 10px;
text-align: center;
border: 1px solid #000000;
border-radius: .25rem;
}

.image{
margin:0 auto;
}

</style>

# Rapidata Flux 2 Pro Preference

<a href="https://www.rapidata.ai">
<img src="https://cdn-uploads.huggingface.co/production/uploads/66f5624c42b853e73e0738eb/jfxR79bOztqaC6_yNNnGU.jpeg" width="400" alt="Dataset visualization">
</a>

This T2I dataset contains over ~400'000 human responses from over ~50'000 individual annotators, collected in less than 7h using the [Rapidata Python API](https://docs.rapidata.ai), accessible to anyone and ideal for large scale evaluation.
Evaluating Flux 2 Pro (version from 25.11.25) across three categories: preference, coherence, and alignment.

Explore our latest model rankings on our [website](https://www.rapidata.ai/benchmark).

If you get value from this dataset and would like to see more in the future, please consider liking it ❤️

To add your own model to the benchmark send us an e-mail at: [e-mail]

## Overview

The evaluation consists of 1v1 comparisons between Flux 2 Pro (version from 24.7.2025) and 18 other models:
- 4o
- Flux-1-pro
- Flux-1.1-pro
- imagen 4 ultra
- Aurora
- Imagen-3
- DALL-E 3
- Midjourney-5.2
- Frames-23-1-25
- Stable Diffusion 3
- Janus-7b.
- hidream-l1-full
- Recraft V2
- Ideogram V2
- halfmoon-4-4-25
- Lumina-15-2-25
- Imagen 4 Ultra 20.5.25
- Imagen 4 Ultra 24.7.25
- Recraft v3
- Hunyuan Image 2.1

## Alignment

The alignment score quantifies how well an video matches its prompt. Users were asked: "Which image matches the description better?".

<div class="vertical-container">
<div class="container">
<div class="text-center">
<q>The cracked rectangle was leaning against the glossy cylinder and the peeling triangle.</q>
</div>
<div class="image-container">
<div>
<h3 class="score-amount">Flux 2 Pro </h3>
<div class="score-percentage">Score: 100%</div>
<img style="border: 5px solid #18c54f;" src="https://cdn-uploads.huggingface.co/production/uploads/672b7d79fd1e92e3c3567435/vo7NdtZ6V1x0hVD_piOle.jpeg" width=500>
</div>
<div>
<h3 class="score-amount">Seedream 3 </h3>
<div class="score-percentage">Score: 0%</div>
<img src="https://cdn-uploads.huggingface.co/production/uploads/672b7d79fd1e92e3c3567435/psHazJXGQ6IN1rOakPZDO.jpeg" width=500>
</div>
</div>
</div>

<div class="container">
<div class="text-center">
<q>The square coaster was next to the circular glass.</q>
</div>
<div class="image-container">
<div>
<h3 class="score-amount">Flux 2 Pro </h3>
<div class="score-percentage">Score: 0%</div>
<img src="https://cdn-uploads.huggingface.co/production/uploads/672b7d79fd1e92e3c3567435/I5kI3eurHZSqWRZay-su0.jpeg" width=500>
</div>
<div>
<h3 class="score-amount">4o </h3>
<div class="score-percentage">Score: 100%</div>
<img style="border: 5px solid #18c54f;" src="https://cdn-uploads.huggingface.co/production/uploads/672b7d79fd1e92e3c3567435/Fj8QL93Nokm_LgJoAtlSk.jpeg" width=500>
</div>
</div>
</div>
</div>

## Coherence

The coherence score measures whether the generated video is logically consistent and free from artifacts or visual glitches. Without seeing the original prompt, users were asked: "Which image has **more** glitches and is **more** likely to be AI generated?"

<div class="vertical-container">
<div class="container">
<div class="image-container">
<div>
<h3 class="score-amount">Flux 2 Pro </h3>
<div class="score-percentage">Glitch Rating: 0%</div>
<img style="border: 5px solid #18c54f;" src="https://cdn-uploads.huggingface.co/production/uploads/672b7d79fd1e92e3c3567435/cdR8kXoWeJOFhwsJSE8-8.jpeg" width=500>
</div>
<div>
<h3 class="score-amount">Flux 1 Pro </h3>
<div class="score-percentage">Glitch Rating: 100%</div>
<img src="https://cdn-uploads.huggingface.co/production/uploads/672b7d79fd1e92e3c3567435/QvFsh1iMX6sBBc_xox_gG.jpeg" width=500>
</div>
</div>
</div>
<div class="container">
<div class="image-container">
<div>
<h3 class="score-amount">Flux 2 Pro </h3>
<div class="score-percentage">Glitch Rating: 100%</div>
<img src="https://cdn-uploads.huggingface.co/production/uploads/672b7d79fd1e92e3c3567435/5kvf9hSGUApakuMaTMTwu.jpeg" width=500>
</div>
<div>
<h3 class="score-amount">Halfmoon 4 </h3>
<div class="score-percentage">Glitch Rating: 0%</div>
<img style="border: 5px solid #18c54f;" src="https://cdn-uploads.huggingface.co/production/uploads/672b7d79fd1e92e3c3567435/WzoqlDL-SDBYTTwOR1bAF.jpeg" width=500>
</div>
</div>
</div>
</div>

## Preference

The preference score reflects how visually appealing participants found each image, independent of the prompt. Users were asked: "Which image do you prefer?"

<div class="vertical-container">
<div class="container">
<div class="image-container">
<div>
<h3 class="score-amount">Flux 2 Pro </h3>
<div class="score-percentage">Score: 100%</div>
<img style="border: 5px solid #18c54f;" src="https://cdn-uploads.huggingface.co/production/uploads/672b7d79fd1e92e3c3567435/LOb8N1QYMMrgwK3Jg4KQj.jpeg" width=500>
</div>
<div>
<h3 class="score-amount">Stable Diffusion 3 </h3>
<div class="score-percentage">Score: 0%</div>
<img src="https://cdn-uploads.huggingface.co/production/uploads/672b7d79fd1e92e3c3567435/yT5LbhwZrOmU4AeeXE49c.jpeg" width=500>
</div>
</div>
</div>
<div class="container">
<div class="image-container">
<div>
<h3 class="score-amount">Flux 2 Pro </h3>
<div class="score-percentage">Score: 0%</div>
<img src="https://cdn-uploads.huggingface.co/production/uploads/672b7d79fd1e92e3c3567435/6wfRY8v6svdIS85uUmvaB.jpeg" width=500>
</div>
<div>
<h3 class="score-amount">Lumina </h3>
<div class="score-percentage">Score: 100%</div>
<img style="border: 5px solid #18c54f;" src="https://cdn-uploads.huggingface.co/production/uploads/672b7d79fd1e92e3c3567435/jQxurLywO5lYziJ82bVl2.jpeg" width=500>
</div>
</div>
</div>
</div>

## Benchmark

<a href="https://app.rapidata.ai/mri/benchmarks/686e5afa75adbe4a56f90549">
<div class="link-container">
<div>
Check out the Benchmark!
</div>
<img class="image" src="https://cdn-uploads.huggingface.co/production/uploads/672b7d79fd1e92e3c3567435/HyeJQ_Pt_K4jkySp55IGO.png" alt="Audio Benchmark">
</div>
</a>

## About Rapidata

Rapidata's technology makes collecting human feedback at scale faster and more accessible than ever before. Visit [rapidata.ai](https://www.rapidata.ai/) to learn more about how we're revolutionizing human feedback collection for AI development.