Recraft-V2_t2i_human_preference

<style>

.vertical-container {
display: flex;
flex-direction: column;
gap: 60px;
}

.image-container img {
max-height: 250px; /* Set the desired height */
margin:0;
object-fit: contain; /* Ensures the aspect ratio is maintained */
width: auto; /* Adjust width automatically based on height */
box-sizing: content-box;
}

.image-container {
display: flex; /* Aligns images side by side */
justify-content: space-around; /* Space them evenly */
align-items: center; /* Align them vertically */
gap: .5rem
}

.container {
width: 90%;
margin: 0 auto;
}

.text-center {
text-align: center;
}

.score-amount {
margin: 0;
margin-top: 10px;
}

.score-percentage {Score:
font-size: 12px;
font-weight: semi-bold;
}

</style>

# Rapidata Recraft-V2 Preference

<a href="https://www.rapidata.ai">
<img src="https://cdn-uploads.huggingface.co/production/uploads/66f5624c42b853e73e0738eb/jfxR79bOztqaC6_yNNnGU.jpeg" width="400" alt="Dataset visualization">
</a>

This T2I dataset contains over 195k human responses from over 47k individual annotators, collected in just ~1 Day using the [Rapidata Python API](https://docs.rapidata.ai), accessible to anyone and ideal for large scale evaluation.
Evaluating Recraft-V2 across three categories: preference, coherence, and alignment.

Explore our latest model rankings on our [website](https://www.rapidata.ai/benchmark).

If you get value from this dataset and would like to see more in the future, please consider liking it.

## Overview

This T2I dataset contains over 195k human responses from over 47k individual annotators, collected in just ~1 Day.
Evaluating Recraft-V2 across three categories: preference, coherence, and alignment.

The evaluation consists of 1v1 comparisons between Recraft V2 and 10 other models: Lumina-15-2-25, Imagen-3, Flux-1.1-pro, Flux-1-pro, DALL-E 3, Midjourney-5.2, Stable Diffusion 3, Aurora and Janus-7b.

## Alignment

The alignment score quantifies how well an video matches its prompt. Users were asked: "Which image matches the description better?".

<div class="vertical-container">
<div class="container">
<div class="text-center">
<q>A person and a airplane, the person is bigger than the airplane.</q>
</div>
<div class="image-container">
<div>
<h3 class="score-amount">Recraft V2 </h3>
<div class="score-percentage">Score: 91.6%</div>
<img style="border: 3px solid #18c54f;" src="https://cdn-uploads.huggingface.co/production/uploads/664dcc6296d813a7e15e170e/ErVTBXcjUxhb50NhBWp6H.jpeg" width=500>
</div>
<div>
<h3 class="score-amount">Flux-1.1 Pro </h3>
<div class="score-percentage">Score: 8.4%</div>
<img src="https://cdn-uploads.huggingface.co/production/uploads/664dcc6296d813a7e15e170e/LO46J3T16njt3pbFlOdPc.jpeg" width=500>
</div>
</div>
</div>

<div class="container">
<div class="text-center">
<q>Four dogs on the street.</q>
</div>
<div class="image-container">
<div>
<h3 class="score-amount">Recraft V2</h3>
<div class="score-percentage">Score: 0%</div>
<img src="https://cdn-uploads.huggingface.co/production/uploads/664dcc6296d813a7e15e170e/YEmXF0lUrwLnRWlIRfnKW.jpeg" width=500>
</div>
<div>
<h3 class="score-amount">Lumina-15-2-25</h3>
<div class="score-percentage">Score: 100%</div>
<img style="border: 3px solid #18c54f;" src="https://cdn-uploads.huggingface.co/production/uploads/664dcc6296d813a7e15e170e/KuE1LS4yJRcSkZigstF_i.jpeg" width=500>
</div>
</div>
</div>
</div>

## Coherence

The coherence score measures whether the generated video is logically consistent and free from artifacts or visual glitches. Without seeing the original prompt, users were asked: "Which image has **more** glitches and is **more** likely to be AI generated?"

<div class="vertical-container">
<div class="container">
<div class="image-container">
<div>
<h3 class="score-amount">Recraft V2 </h3>
<div class="score-percentage">Glitch Rating: 0%</div>
<img style="border: 3px solid #18c54f;" src="https://cdn-uploads.huggingface.co/production/uploads/664dcc6296d813a7e15e170e/xW1D3Z02doO8Fjtf95GgW.jpeg" width=500>
</div>
<div>
<h3 class="score-amount">Janus-7B </h3>
<div class="score-percentage">Glitch Rating: 100%</div>
<img src="https://cdn-uploads.huggingface.co/production/uploads/664dcc6296d813a7e15e170e/_pD7I7Ytg612EdjY0Ccqg.jpeg" width=500>
</div>
</div>
</div>

<div class="container">
<div class="image-container">
<div>
<h3 class="score-amount">Recraft V2 </h3>
<div class="score-percentage">Glitch Rating: 90.2%</div>
<img src="https://cdn-uploads.huggingface.co/production/uploads/664dcc6296d813a7e15e170e/bnDvn3RH1kSA4wsu2tW-H.jpeg" width=500>
</div>
<div>
<h3 class="score-amount">Frames-23-1-25</h3>
<div class="score-percentage">Glitch Rating: 9.8%</div>
<img style="border: 3px solid #18c54f;" src="https://cdn-uploads.huggingface.co/production/uploads/664dcc6296d813a7e15e170e/tOIQ3BGH3ak7VGypR4ERo.jpeg" width=500>
</div>
</div>
</div>
</div>

## Preference

The preference score reflects how visually appealing participants found each image, independent of the prompt. Users were asked: "Which image do you prefer?"

<div class="vertical-container">
<div class="container">
<div class="image-container">
<div>
<h3 class="score-amount">Recraft V2</h3>
<div class="score-percentage">Score: 100%</div>
<img style="border: 3px solid #18c54f;" src="https://cdn-uploads.huggingface.co/production/uploads/664dcc6296d813a7e15e170e/tFyjXMgsohsNXYmkz74jW.jpeg" width=500>
</div>
<div>
<h3 class="score-amount">Lumina-15-2-25</h3>
<div class="score-percentage">Score: 0%</div>
<img src="https://cdn-uploads.huggingface.co/production/uploads/664dcc6296d813a7e15e170e/DgtW_5bk-Q1p_KsZIHaOE.jpeg" width=500>
</div>
</div>
</div>

<div class="container">
<div class="image-container">
<div>
<h3 class="score-amount">Recraft V2 </h3>
<div class="score-percentage">Score: 0%</div>
<img src="https://cdn-uploads.huggingface.co/production/uploads/664dcc6296d813a7e15e170e/0kv207jq2I0ZAe5zkcOCq.jpeg" width=500>
</div>
<div>
<h3 class="score-amount">Frames-23-1-25 </h3>
<div class="score-percentage">Score: 100%</div>
<img style="border: 3px solid #18c54f;" src="https://cdn-uploads.huggingface.co/production/uploads/664dcc6296d813a7e15e170e/KUC7bX--M7Hx8BTO4Ns0c.jpeg" width=500>
</div>
</div>
</div>
</div>

## About Rapidata

Rapidata's technology makes collecting human feedback at scale faster and more accessible than ever before. Visit [rapidata.ai](https://www.rapidata.ai/) to learn more about how we're revolutionizing human feedback collection for AI development.

<style>

.vertical-container {
display: flex;
flex-direction: column;
gap: 60px;
}

.image-container img {
max-height: 250px; /* Set the desired height */
margin:0;
object-fit: contain; /* Ensures the aspect ratio is maintained */
width: auto; /* Adjust width automatically based on height */
box-sizing: content-box;
}

.image-container {
display: flex; /* Aligns images side by side */
justify-content: space-around; /* Space them evenly */
align-items: center; /* Align them vertically */
gap: .5rem
}

.container {
width: 90%;
margin: 0 auto;
}

.text-center {
text-align: center;
}

.score-amount {
margin: 0;
margin-top: 10px;
}

.score-percentage {Score:
font-size: 12px;
font-weight: semi-bold;
}

</style>

# Rapidata Recraft-V2 偏好数据集

<a href="https://www.rapidata.ai">
<img src="https://cdn-uploads.huggingface.co/production/uploads/66f5624c42b853e73e0738eb/jfxR79bOztqaC6_yNNnGU.jpeg" width="400" alt="Dataset visualization">
</a>

该文本到图像（Text-to-Image, T2I）数据集依托Rapidata Python API，仅用约1天便收集到来自4.7万余名独立标注者的19.5万余条人类标注反馈，全量对外开放，是大规模模型评估的理想之选。
本次评估围绕偏好性、连贯性与对齐性三大维度对Recraft-V2模型展开测试。

可前往我们的官网[https://www.rapidata.ai](https://www.rapidata.ai)查看最新的模型排名榜单。

若您从本数据集获益并希望后续获取更多同类资源，欢迎为其点赞。

## 数据集概览

本文本到图像数据集共收集到来自4.7万余名独立标注者的19.5万余条人类标注反馈，仅耗时约1天完成采集。
本次评估围绕偏好性、连贯性与对齐性三大维度对Recraft-V2模型进行测试。

评估采用1v1对比形式，将Recraft V2与其余10款模型进行对标，分别为：Lumina-15-2-25、Imagen-3、Flux-1.1-pro、Flux-1-pro、DALL-E 3、Midjourney-5.2、Stable Diffusion 3、Aurora以及Janus-7b。

## 对齐性评估

对齐性得分用于衡量生成图像与输入提示词的匹配程度。本次调研向标注者提出问题：“哪张图像更贴合描述内容？”

<div class="vertical-container">
<div class="container">
<div class="text-center">
<q>一个人与一架飞机，且人比飞机更大。</q>
</div>
<div class="image-container">
<div>
<h3 class="score-amount">Recraft V2 </h3>
<div class="score-percentage">得分：91.6%</div>
<img style="border: 3px solid #18c54f;" src="https://cdn-uploads.huggingface.co/production/uploads/664dcc6296d813a7e15e170e/ErVTBXcjUxhb50NhBWp6H.jpeg" width=500>
</div>
<div>
<h3 class="score-amount">Flux-1.1 Pro </h3>
<div class="score-percentage">得分：8.4%</div>
<img src="https://cdn-uploads.huggingface.co/production/uploads/664dcc6296d813a7e15e170e/LO46J3T16njt3pbFlOdPc.jpeg" width=500>
</div>
</div>
</div>

<div class="container">
<div class="text-center">
<q>街道上的四只狗。</q>
</div>
<div class="image-container">
<div>
<h3 class="score-amount">Recraft V2</h3>
<div class="score-percentage">得分：0%</div>
<img src="https://cdn-uploads.huggingface.co/production/uploads/664dcc6296d813a7e15e170e/YEmXF0lUrwLnRWlIRfnKW.jpeg" width=500>
</div>
<div>
<h3 class="score-amount">Lumina-15-2-25</h3>
<div class="score-percentage">得分：100%</div>
<img style="border: 3px solid #18c54f;" src="https://cdn-uploads.huggingface.co/production/uploads/664dcc6296d813a7e15e170e/KuE1LS4yJRcSkZigstF_i.jpeg" width=500>
</div>
</div>
</div>
</div>

## 连贯性评估

连贯性得分用于评估生成图像的逻辑自洽性，以及是否存在伪影或视觉瑕疵。本次调研不向标注者展示原始提示词，仅提问：“哪张图像存在**更多**瑕疵，且更有可能是AI生成？”

<div class="vertical-container">
<div class="container">
<div class="image-container">
<div>
<h3 class="score-amount">Recraft V2 </h3>
<div class="score-percentage">瑕疵评分：0%</div>
<img style="border: 3px solid #18c54f;" src="https://cdn-uploads.huggingface.co/production/uploads/664dcc6296d813a7e15e170e/xW1D3Z02doO8Fjtf95GgW.jpeg" width=500>
</div>
<div>
<h3 class="score-amount">Janus-7B </h3>
<div class="score-percentage">瑕疵评分：100%</div>
<img src="https://cdn-uploads.huggingface.co/production/uploads/664dcc6296d813a7e15e170e/_pD7I7Ytg612EdjY0Ccqg.jpeg" width=500>
</div>
</div>
</div>

<div class="container">
<div class="image-container">
<div>
<h3 class="score-amount">Recraft V2 </h3>
<div class="score-percentage">瑕疵评分：90.2%</div>
<img src="https://cdn-uploads.huggingface.co/production/uploads/664dcc6296d813a7e15e170e/bnDvn3RH1kSA4wsu2tW-H.jpeg" width=500>
</div>
<div>
<h3 class="score-amount">Frames-23-1-25</h3>
<div class="score-percentage">瑕疵评分：9.8%</div>
<img style="border: 3px solid #18c54f;" src="https://cdn-uploads.huggingface.co/production/uploads/664dcc6296d813a7e15e170e/tOIQ3BGH3ak7VGypR4ERo.jpeg" width=500>
</div>
</div>
</div>
</div>

## 偏好性评估

偏好性得分用于体现标注者对单张图像的视觉审美偏好，与原始提示词无关。本次调研向标注者提问：“你更偏好哪张图像？”

<div class="vertical-container">
<div class="container">
<div class="image-container">
<div>
<h3 class="score-amount">Recraft V2</h3>
<div class="score-percentage">得分：100%</div>
<img style="border: 3px solid #18c54f;" src="https://cdn-uploads.huggingface.co/production/uploads/664dcc6296d813a7e15e170e/tFyjXMgsohsNXYmkz74jW.jpeg" width=500>
</div>
<div>
<h3 class="score-amount">Lumina-15-2-25</h3>
<div class="score-percentage">得分：0%</div>
<img src="https://cdn-uploads.huggingface.co/production/uploads/664dcc6296d813a7e15e170e/DgtW_5bk-Q1p_KsZIHaOE.jpeg" width=500>
</div>
</div>
</div>

<div class="container">
<div class="image-container">
<div>
<h3 class="score-amount">Recraft V2 </h3>
<div class="score-percentage">得分：0%</div>
<img src="https://cdn-uploads.huggingface.co/production/uploads/664dcc6296d813a7e15e170e/0kv207jq2I0ZAe5zkcOCq.jpeg" width=500>
</div>
<div>
<h3 class="score-amount">Frames-23-1-25 </h3>
<div class="score-percentage">得分：100%</div>
<img style="border: 3px solid #18c54f;" src="https://cdn-uploads.huggingface.co/production/uploads/664dcc6296d813a7e15e170e/KUC7bX--M7Hx8BTO4Ns0c.jpeg" width=500>
</div>
</div>
</div>
</div>

## 关于Rapidata

Rapidata的技术让大规模人类标注反馈的采集工作相较以往更快捷、更普惠。欢迎访问[rapidata.ai](https://www.rapidata.ai/)，了解我们如何革新AI开发领域的人类反馈采集流程。