Recraft-v3-24-7-25_t2i_human_preference

<style>

.vertical-container {
display: flex;
flex-direction: column;
gap: 60px;
}

.horizontal-container {
display: flex;
flex-direction: row;
justify-content: center;
gap: 60px;
}

.image-container img {
max-height: 250px; /* Set the desired height */
margin:0;
object-fit: contain; /* Ensures the aspect ratio is maintained */
width: auto; /* Adjust width automatically based on height */
box-sizing: content-box;
}

.image-container img.big {
max-height: 350px; /* Set the desired height */
}

.image-container {
display: flex; /* Aligns images side by side */
justify-content: space-around; /* Space them evenly */
align-items: center; /* Align them vertically */
gap: .5rem
}

.container {
width: 90%;
margin: 0 auto;
}

.text-center {
text-align: center;
}

.score-amount {
margin: 0;
margin-top: 10px;
}

.score-percentage {Score:
font-size: 12px;
font-weight: semi-bold;
}

</style>

# Rapidata Recraft v3 Preference

<a href="https://www.rapidata.ai">
<img src="https://cdn-uploads.huggingface.co/production/uploads/66f5624c42b853e73e0738eb/jfxR79bOztqaC6_yNNnGU.jpeg" width="400" alt="Dataset visualization">
</a>

This T2I dataset contains over ~400'000 human responses from over ~50'000 individual annotators, collected in less than 7h using the [Rapidata Python API](https://docs.rapidata.ai), accessible to anyone and ideal for large scale evaluation.
Evaluating Recraft v3 (version from 24.7.2025) across three categories: preference, coherence, and alignment.

Explore our latest model rankings on our [website](https://www.rapidata.ai/benchmark).

If you get value from this dataset and would like to see more in the future, please consider liking it ❤️

To add your own model to the benchmark send us an e-mail at: [e-mail]

## Overview

The evaluation consists of 1v1 comparisons between Recraft v3 (version from 24.7.2025) and 18 other models:
- 4o
- Flux-1-pro
- Flux-1.1-pro
- imagen 4 ultra
- Aurora
- Imagen-3
- DALL-E 3
- Midjourney-5.2
- Frames-23-1-25
- Stable Diffusion 3
- Janus-7b.
- hidream-l1-full
- Recraft V2
- Ideogram V2
- halfmoon-4-4-25
- Lumina-15-2-25
- Imagen 4 Ultra 20.5.25
- Imagen 4 Ultra 24.7.25

## Alignment

The alignment score quantifies how well an video matches its prompt. Users were asked: "Which image matches the description better?".

<div class="vertical-container">
<div class="container">
<div class="text-center">
<q>A device consisting of a circular canopy of cloth on a folding metal frame supported by a central rod, used as protection against rain or sometimes sun.</q>
</div>
<div class="image-container">
<div>
<h3 class="score-amount">Recraft v3 </h3>
<div class="score-percentage">Score: 100%</div>
<img style="border: 5px solid #18c54f;" src="https://cdn-uploads.huggingface.co/production/uploads/672b7d79fd1e92e3c3567435/F4s5H4PiV_-dagyacaiFr.jpeg" width=500>
</div>
<div>
<h3 class="score-amount">Janus 7b </h3>
<div class="score-percentage">Score: 0%</div>
<img src="https://cdn-uploads.huggingface.co/production/uploads/672b7d79fd1e92e3c3567435/tFGWSjlnS8W7w4yH9f71Q.jpeg" width=500>
</div>
</div>
</div>

<div class="container">
<div class="text-center">
<q>A brown toilet with a white wooden seat .</q>
</div>
<div class="image-container">
<div>
<h3 class="score-amount">Recraft v3 </h3>
<div class="score-percentage">Score: 0%</div>
<img src="https://cdn-uploads.huggingface.co/production/uploads/672b7d79fd1e92e3c3567435/n00lh_XkDThrxrd2Nlr8g.jpeg" width=500>
</div>
<div>
<h3 class="score-amount">Stable Diffusion 3 </h3>
<div class="score-percentage">Score: 100%</div>
<img style="border: 5px solid #18c54f;" src="https://cdn-uploads.huggingface.co/production/uploads/672b7d79fd1e92e3c3567435/rk2iBGCpVuCf2SWQoWDyR.jpeg" width=500>
</div>
</div>
</div>
</div>

## Coherence

The coherence score measures whether the generated video is logically consistent and free from artifacts or visual glitches. Without seeing the original prompt, users were asked: "Which image has **more** glitches and is **more** likely to be AI generated?"

<div class="vertical-container">
<div class="container">
<div class="image-container">
<div>
<h3 class="score-amount">Recraft v3 </h3>
<div class="score-percentage">Glitch Rating: 0%</div>
<img style="border: 5px solid #18c54f;" src="https://cdn-uploads.huggingface.co/production/uploads/672b7d79fd1e92e3c3567435/CADQvkJBqne_sbfqm0MPF.jpeg" width=500>
</div>
<div>
<h3 class="score-amount">Stable Diffusion 3 </h3>
<div class="score-percentage">Glitch Rating: 100%</div>
<img src="https://cdn-uploads.huggingface.co/production/uploads/672b7d79fd1e92e3c3567435/c7t-Nak8NGA-LAV4rlN0P.jpeg" width=500>
</div>
</div>
</div>

<div class="container">
<div class="image-container">
<div>
<h3 class="score-amount">Recraft v3 </h3>
<div class="score-percentage">Glitch Rating: 100%</div>
<img src="https://cdn-uploads.huggingface.co/production/uploads/672b7d79fd1e92e3c3567435/zTf7fXlzQAySORzUiV0LJ.jpeg" width=500>
</div>
<div>
<h3 class="score-amount">Imagen 3 </h3>
<div class="score-percentage">Glitch Rating: 0%</div>
<img style="border: 5px solid #18c54f;" src="https://cdn-uploads.huggingface.co/production/uploads/672b7d79fd1e92e3c3567435/49S2xgqyqgZjBtoc6eaGY.jpeg" width=500>
</div>
</div>
</div>
</div>

## Preference

The preference score reflects how visually appealing participants found each image, independent of the prompt. Users were asked: "Which image do you prefer?"

<div class="vertical-container">
<div class="container">
<div class="image-container">
<div>
<h3 class="score-amount">Recraft v3 </h3>
<div class="score-percentage">Score: 100%</div>
<img style="border: 5px solid #18c54f;" src="https://cdn-uploads.huggingface.co/production/uploads/672b7d79fd1e92e3c3567435/kB_ldhmftj7xf0R4RMQwE.jpeg" width=500>
</div>
<div>
<h3 class="score-amount">Lumina-15-2-25</h3>
<div class="score-percentage">Score: 0%</div>
<img src="https://cdn-uploads.huggingface.co/production/uploads/672b7d79fd1e92e3c3567435/bisy3ru4WA8_KNC5cPHCB.jpeg" width=500>
</div>
</div>
</div>

<div class="container">
<div class="image-container">
<div>
<h3 class="score-amount">Recraft v3 </h3>
<div class="score-percentage">Score: 0%</div>
<img src="https://cdn-uploads.huggingface.co/production/uploads/672b7d79fd1e92e3c3567435/E9AB5ihYHE-0n2D2sp_2z.jpeg" width=500>
</div>
<div>
<h3 class="score-amount">Flux 1 Pro </h3>
<div class="score-percentage">Score: 100%</div>
<img style="border: 5px solid #18c54f;" src="https://cdn-uploads.huggingface.co/production/uploads/672b7d79fd1e92e3c3567435/hisW7cK98yx5miz6FcwOg.jpeg" width=500>
</div>
</div>
</div>
</div>

## About Rapidata

Rapidata's technology makes collecting human feedback at scale faster and more accessible than ever before. Visit [rapidata.ai](https://www.rapidata.ai/) to learn more about how we're revolutionizing human feedback collection for AI development.

<style>

.vertical-container {
display: flex;
flex-direction: column;
gap: 60px;
}

.horizontal-container {
display: flex;
flex-direction: row;
justify-content: center;
gap: 60px;
}

.image-container img {
max-height: 250px; /* Set the desired height */
margin:0;
object-fit: contain; /* Ensures the aspect ratio is maintained */
width: auto; /* Adjust width automatically based on height */
box-sizing: content-box;
}

.image-container img.big {
max-height: 350px; /* Set the desired height */
}

.image-container {
display: flex; /* Aligns images side by side */
justify-content: space-around; /* Space them evenly */
align-items: center; /* Align them vertically */
gap: .5rem
}

.container {
width: 90%;
margin: 0 auto;
}

.text-center {
text-align: center;
}

.score-amount {
margin: 0;
margin-top: 10px;
}

.score-percentage {
font-size: 12px;
font-weight: semi-bold;
}

</style>

# Rapidata Recraft v3 偏好数据集

<a href="https://www.rapidata.ai">
<img src="https://cdn-uploads.huggingface.co/production/uploads/66f5624c42b853e73e0738eb/jfxR79bOztqaC6_yNNnGU.jpeg" width="400" alt="数据集可视化">
</a>

本文本到图像（Text-to-Image, T2I）数据集包含超40万条人类标注反馈，由超过5万名独立标注员完成，通过[Rapidata Python应用程序接口（Rapidata Python API）](https://docs.rapidata.ai)在7小时内采集完成，面向所有用户开放，非常适合大规模模型评估。
本次评估针对2025年7月24日版本的Recraft v3模型，从偏好性、连贯性与对齐性三个维度展开。

可前往我们的[官方网站](https://www.rapidata.ai)查看最新的模型排名。

若您从本数据集获益并希望后续获取更多同类资源，欢迎点赞支持 ❤️

若您希望将自有模型加入基准测试，请发送邮件至：[e-mail]

## 数据集概览
本次评估采用1v1对比形式，将2025年7月24日版本的Recraft v3模型与其余18款模型进行比对，具体包括：
- 4o
- Flux-1-pro
- Flux-1.1-pro
- Imagen 4 Ultra
- Aurora
- Imagen-3
- DALL-E 3
- Midjourney-5.2
- Frames-23-1-25
- Stable Diffusion 3
- Janus-7b
- hidream-l1-full
- Recraft V2
- Ideogram V2
- halfmoon-4-4-25
- Lumina-15-2-25
- Imagen 4 Ultra 20.5.25
- Imagen 4 Ultra 24.7.25

## 对齐性
对齐性得分用于量化生成图像与输入提示词的匹配程度。标注员被问及：「哪张图像更贴合给定描述？」

<div class="vertical-container">
<div class="container">
<div class="text-center">
<q>一种由中心杆支撑的折叠金属框架搭配圆形布质顶篷组成的装置，用于遮阳避雨。</q>
</div>
<div class="image-container">
<div>
<h3 class="score-amount">Recraft v3 </h3>
<div class="score-percentage">得分：100%</div>
<img style="border: 5px solid #18c54f;" src="https://cdn-uploads.huggingface.co/production/uploads/672b7d79fd1e92e3c3567435/F4s5H4PiV_-dagyacaiFr.jpeg" width=500>
</div>
<div>
<h3 class="score-amount">Janus 7b </h3>
<div class="score-percentage">得分：0%</div>
<img src="https://cdn-uploads.huggingface.co/production/uploads/672b7d79fd1e92e3c3567435/tFGWSjlnS8W7w4yH9f71Q.jpeg" width=500>
</div>
</div>
</div>

<div class="container">
<div class="text-center">
<q>一款带有白色木质座圈的棕色马桶。</q>
</div>
<div class="image-container">
<div>
<h3 class="score-amount">Recraft v3 </h3>
<div class="score-percentage">得分：0%</div>
<img src="https://cdn-uploads.huggingface.co/production/uploads/672b7d79fd1e92e3c3567435/n00lh_XkDThrxrd2Nlr8g.jpeg" width=500>
</div>
<div>
<h3 class="score-amount">Stable Diffusion 3 </h3>
<div class="score-percentage">得分：100%</div>
<img style="border: 5px solid #18c54f;" src="https://cdn-uploads.huggingface.co/production/uploads/672b7d79fd1e92e3c3567435/rk2iBGCpVuCf2SWQoWDyR.jpeg" width=500>
</div>
</div>
</div>
</div>

## 连贯性
连贯性得分用于衡量生成图像是否具备逻辑一致性，且无人工痕迹或视觉瑕疵。在不查看原始提示词的前提下，标注员被问及：「哪张图像存在**更多**瑕疵，且**更有可能**是AI生成作品？」

<div class="vertical-container">
<div class="container">
<div class="image-container">
<div>
<h3 class="score-amount">Recraft v3 </h3>
<div class="score-percentage">瑕疵评级：0%</div>
<img style="border: 5px solid #18c54f;" src="https://cdn-uploads.huggingface.co/production/uploads/672b7d79fd1e92e3c3567435/CADQvkJBqne_sbfqm0MPF.jpeg" width=500>
</div>
<div>
<h3 class="score-amount">Stable Diffusion 3 </h3>
<div class="score-percentage">瑕疵评级：100%</div>
<img src="https://cdn-uploads.huggingface.co/production/uploads/672b7d79fd1e92e3c3567435/c7t-Nak8NGA-LAV4rlN0P.jpeg" width=500>
</div>
</div>
</div>

<div class="container">
<div class="image-container">
<div>
<h3 class="score-amount">Recraft v3 </h3>
<div class="score-percentage">瑕疵评级：100%</div>
<img src="https://cdn-uploads.huggingface.co/production/uploads/672b7d79fd1e92e3c3567435/zTf7fXlzQAySORzUiV0LJ.jpeg" width=500>
</div>
<div>
<h3 class="score-amount">Imagen 3 </h3>
<div class="score-percentage">瑕疵评级：0%</div>
<img style="border: 5px solid #18c54f;" src="https://cdn-uploads.huggingface.co/production/uploads/672b7d79fd1e92e3c3567435/49S2xgqyqgZjBtoc6eaGY.jpeg" width=500>
</div>
</div>
</div>
</div>

## 偏好性
偏好性得分用于反映参与者对每张图像的视觉喜好程度，不受提示词限制。标注员被问及：「你更偏好哪张图像？」

<div class="vertical-container">
<div class="container">
<div class="image-container">
<div>
<h3 class="score-amount">Recraft v3 </h3>
<div class="score-percentage">得分：100%</div>
<img style="border: 5px solid #18c54f;" src="https://cdn-uploads.huggingface.co/production/uploads/672b7d79fd1e92e3c3567435/kB_ldhmftj7xf0R4RMQwE.jpeg" width=500>
</div>
<div>
<h3 class="score-amount">Lumina-15-2-25</h3>
<div class="score-percentage">得分：0%</div>
<img src="https://cdn-uploads.huggingface.co/production/uploads/672b7d79fd1e92e3c3567435/bisy3ru4WA8_KNC5cPHCB.jpeg" width=500>
</div>
</div>
</div>

<div class="container">
<div class="image-container">
<div>
<h3 class="score-amount">Recraft v3 </h3>
<div class="score-percentage">得分：0%</div>
<img src="https://cdn-uploads.huggingface.co/production/uploads/672b7d79fd1e92e3c3567435/E9AB5ihYHE-0n2D2sp_2z.jpeg" width=500>
</div>
<div>
<h3 class="score-amount">Flux 1 Pro </h3>
<div class="score-percentage">得分：100%</div>
<img style="border: 5px solid #18c54f;" src="https://cdn-uploads.huggingface.co/production/uploads/672b7d79fd1e92e3c3567435/hisW7cK98yx5miz6FcwOg.jpeg" width=500>
</div>
</div>
</div>
</div>

## 关于Rapidata
Rapidata的技术让大规模人类反馈采集比以往任何时候都更快捷、更易用。请访问[rapidata.ai](https://www.rapidata.ai/)了解我们如何革新AI开发中的人类反馈采集流程。