Rapidata/Face_Generation_Benchmark

---
dataset_info:
features:
- name: prompt
dtype: string
- name: image1
dtype: image
- name: image2
dtype: image
- name: model1
dtype: string
- name: model2
dtype: string
- name: weighted_score_image1
dtype: float32
- name: weighted_score_image2
dtype: float32
- name: detailed_results
dtype: string
splits:
- name: train
num_bytes: 1691529922.55
num_examples: 2310
download_size: 1513490728
dataset_size: 1691529922.55
configs:
- config_name: default
data_files:
- split: train
path: data/train-*
license: cdla-permissive-2.0
task_categories:
- text-to-image
- image-to-text
- image-classification
- reinforcement-learning
language:
- en
tags:
- Human
- Preference
- Coherence
- Alignment
- country
- language
- bytedance
- seedream-3
- qwen
- qwen-image
- recraft-ai
- recraft-v3
- minimax
- image-01
- google
- imagen-4-fast
- black-forest-labs
- flux
- prunaai
- wan-2.2-image
- ideogram-ai
- ideogram
- luma
- photon
- bria
- image-3.2
- luma
- photon-flash
- black-forest-labs
- flux
size_categories:
- 10K<n<100K
pretty_name: Human Face Generation Alignment
---

# Rapidata Human Face Generation Alignment

<a href="https://www.rapidata.ai">
<img src="https://cdn-uploads.huggingface.co/production/uploads/66f5624c42b853e73e0738eb/jfxR79bOztqaC6_yNNnGU.jpeg" width="400" alt="Dataset visualization">
</a>

This T2I dataset contains over ~22'000 human responses, collected in less than 1h using the [Rapidata Python API](https://docs.rapidata.ai), accessible to anyone and ideal for large scale evaluation.
Evaluating 12 different image generation models on which one can generate faces more accurately.

The question that the annotators get asked is: "Which Image follows the description of the human better?"

<strong>To evaluate your own models and create leaderboard check out our [MRI](https://docs.rapidata.ai/mri/) </strong>

## The resulting leaderboard

Checkout the full leaderboard here: https://app.rapidata.ai/mri/benchmarks/68af24ae74482280b62f7596

<a href="https://app.rapidata.ai/mri/benchmarks/68af24ae74482280b62f7596">
<img src="https://cdn-uploads.huggingface.co/production/uploads/66f5624c42b853e73e0738eb/ABLvkbrkTYBNRzipiFJNs.png" alt="Leaderboard">
</a>

| | name | wins | total_matches | score |
|----|-----------------------------------|------|---------------|------------|
| 0 | bytedance/seedream-3 | 2604 | 4472 | 1123.630000|
| 1 | qwen/qwen-image | 1654 | 2877 | 1110.540000|
| 2 | recraft-ai/recraft-v3 | 1157 | 2028 | 1105.070000|
| 3 | minimax/image-01 | 2726 | 5125 | 1051.620000|
| 4 | google/imagen-4-fast | 3205 | 6180 | 1027.690000|
| 5 | black-forest-labs/flux-1.1-pro | 1272 | 2650 | 969.560000 |
| 6 | prunaai/wan-2.2-image | 1121 | 2344 | 967.050000 |
| 7 | ideogram-ai/ideogram-v3-balanced | 2868 | 6170 | 962.890000 |
| 8 | luma/photon | 1839 | 3934 | 955.450000 |
| 9 | bria/image-3.2 | 1507 | 3226 | 951.530000 |
| 10 | luma/photon-flash | 1499 | 3529 | 889.910000 |
| 11 | black-forest-labs/flux-schnell | 1024 | 2417 | 885.060000 |