Imagen4_t2i_human_preference

<style>

.vertical-container {
display: flex;
flex-direction: column;
gap: 60px;
}

.image-container img {
max-height: 250px; /* Set the desired height */
margin:0;
object-fit: contain; /* Ensures the aspect ratio is maintained */
width: auto; /* Adjust width automatically based on height */
box-sizing: content-box;
}

.image-container {
display: flex; /* Aligns images side by side */
justify-content: space-around; /* Space them evenly */
align-items: center; /* Align them vertically */
gap: .5rem
}

.container {
width: 90%;
margin: 0 auto;
}

.text-center {
text-align: center;
}

.score-amount {
margin: 0;
margin-top: 10px;
}

.score-percentage {Score:
font-size: 12px;
font-weight: semi-bold;
}

</style>

# Rapidata Imagen 4 Preference

<a href="https://www.rapidata.ai">
<img src="https://cdn-uploads.huggingface.co/production/uploads/66f5624c42b853e73e0738eb/jfxR79bOztqaC6_yNNnGU.jpeg" width="400" alt="Dataset visualization">
</a>

This T2I dataset contains over 195k human responses from over 70k individual annotators, collected in just ~1 Day using the [Rapidata Python API](https://docs.rapidata.ai), accessible to anyone and ideal for large scale evaluation.
Evaluating Imagen 4 (imagen-4.0-ultra-generate-exp-05-20) across three categories: preference, coherence, and alignment.

Explore our latest model rankings on our [website](https://www.rapidata.ai/benchmark).

If you get value from this dataset and would like to see more in the future, please consider liking it ❤️

## Overview

This T2I dataset contains over 195k human responses from over 70k individual annotators, collected in just ~1 Day.
Evaluating Imagen 4 (imagen-4.0-ultra-generate-exp-05-20) across three categories: preference, coherence, and alignment.

The evaluation consists of 1v1 comparisons between Imagen 4 (imagen-4.0-ultra-generate-exp-05-20) and 15 other models: Hidream I1 full, Halfmoon-4-4-2025, OpenAI 4o-26-3-25, Ideogram V2, Recraft V2, Lumina-15-2-25, Frames-23-1-25, Imagen-3, Flux-1.1-pro, Flux-1-pro, DALL-E 3, Midjourney-5.2, Stable Diffusion 3, Aurora and Janus-7b.
> **Note:** The number following the model name (e.g., Halfmoon-4-4-2025) represents the date (April 4, 2025) on which the images were generated to give an understanding of what model version was used.
## Alignment

The alignment score quantifies how well an video matches its prompt. Users were asked: "Which image matches the description better?".

<div class="vertical-container">
<div class="container">
<div class="text-center">
<q>A blue cup and a green cell phone.</q>
</div>
<div class="image-container">
<div>
<h3 class="score-amount">Imagen 4 </h3>
<div class="score-percentage">Score: 100%</div>
<img style="border: 3px solid #18c54f;" src="https://cdn-uploads.huggingface.co/production/uploads/664dcc6296d813a7e15e170e/8JcDYOeviHCwVQo0vLMB-.jpeg" width=500>
</div>
<div>
<h3 class="score-amount">Frames-23-1-25 </h3>
<div class="score-percentage">Score: 0%</div>
<img src="https://cdn-uploads.huggingface.co/production/uploads/664dcc6296d813a7e15e170e/WmHuNZY4M_NWUMSTSofNL.jpeg" width=500>
</div>
</div>
</div>

<div class="container">
<div class="text-center">
<q>A person is walking with a guidebook and taking a tour of a historic site.</q>
</div>
<div class="image-container">
<div>
<h3 class="score-amount">Imagen 4</h3>
<div class="score-percentage">Score: 0%</div>
<img src="https://cdn-uploads.huggingface.co/production/uploads/664dcc6296d813a7e15e170e/RiyfAra-C5QcCT9lDiohg.jpeg" width=500>
</div>
<div>
<h3 class="score-amount">Aurora</h3>
<div class="score-percentage">Score: 100%</div>
<img style="border: 3px solid #18c54f;" src="https://cdn-uploads.huggingface.co/production/uploads/664dcc6296d813a7e15e170e/SPbiu_SE8S4vRv6p0OZHo.jpeg" width=500>
</div>
</div>
</div>
</div>

## Coherence

The coherence score measures whether the generated video is logically consistent and free from artifacts or visual glitches. Without seeing the original prompt, users were asked: "Which image has **more** glitches and is **more** likely to be AI generated?"

<div class="vertical-container">
<div class="container">
<div class="image-container">
<div>
<h3 class="score-amount">Imagen 4 </h3>
<div class="score-percentage">Glitch Rating: 0%</div>
<img style="border: 3px solid #18c54f;" src="https://cdn-uploads.huggingface.co/production/uploads/664dcc6296d813a7e15e170e/X2jimr0Icoqb4RlX6jG1j.jpeg" width=500>
</div>
<div>
<h3 class="score-amount">Stable Diffusion 3 </h3>
<div class="score-percentage">Glitch Rating: 100%</div>
<img src="https://cdn-uploads.huggingface.co/production/uploads/664dcc6296d813a7e15e170e/UUllaDc48oxLXxeXbF_CD.jpeg" width=500>
</div>
</div>
</div>

<div class="container">
<div class="image-container">
<div>
<h3 class="score-amount">Imagen 4 </h3>
<div class="score-percentage">Glitch Rating: 100%</div>
<img src="https://cdn-uploads.huggingface.co/production/uploads/664dcc6296d813a7e15e170e/1a5dQzX2XxTLIUcgimocy.jpeg" width=500>
</div>
<div>
<h3 class="score-amount">Ideogram</h3>
<div class="score-percentage">Glitch Rating: 0%</div>
<img style="border: 3px solid #18c54f;" src="https://cdn-uploads.huggingface.co/production/uploads/664dcc6296d813a7e15e170e/pI5B7RxysOcUhVEzdqHFq.jpeg" width=500>
</div>
</div>
</div>
</div>

## Preference

The preference score reflects how visually appealing participants found each image, independent of the prompt. Users were asked: "Which image do you prefer?"

<div class="vertical-container">
<div class="container">
<div class="image-container">
<div>
<h3 class="score-amount">Imagen 4</h3>
<div class="score-percentage">Score: 81.83%</div>
<img style="border: 3px solid #18c54f;" src="https://cdn-uploads.huggingface.co/production/uploads/664dcc6296d813a7e15e170e/b3ny4C5fRdfnoIlK3ocFi.jpeg" width=500>
</div>
<div>
<h3 class="score-amount">Frames-23-1-25</h3>
<div class="score-percentage">Score: 18.17%</div>
<img src="https://cdn-uploads.huggingface.co/production/uploads/664dcc6296d813a7e15e170e/KDJSo6s6FMKpXiovP03cJ.jpeg" width=500>
</div>
</div>
</div>

<div class="container">
<div class="image-container">
<div>
<h3 class="score-amount">Imagen 4 </h3>
<div class="score-percentage">Score: 27.42%</div>
<img src="https://cdn-uploads.huggingface.co/production/uploads/664dcc6296d813a7e15e170e/-g7n1pP8QRU0Gcv-iKF4q.jpeg" width=500>
</div>
<div>
<h3 class="score-amount">Frames-23-1-25 </h3>
<div class="score-percentage">Score: 72.58%</div>
<img style="border: 3px solid #18c54f;" src="https://cdn-uploads.huggingface.co/production/uploads/664dcc6296d813a7e15e170e/BMXzpF1loC3wVfWuyj5VX.jpeg" width=500>
</div>
</div>
</div>
</div>

## About Rapidata

Rapidata's technology makes collecting human feedback at scale faster and more accessible than ever before. Visit [rapidata.ai](https://www.rapidata.ai/) to learn more about how we're revolutionizing human feedback collection for AI development.

<style>

.vertical-container {
display: flex;
flex-direction: column;
gap: 60px;
}

.image-container img {
max-height: 250px; /* Set the desired height */
margin:0;
object-fit: contain; /* Ensures the aspect ratio is maintained */
width: auto; /* Adjust width automatically based on height */
box-sizing: content-box;
}

.image-container {
display: flex; /* Aligns images side by side */
justify-content: space-around; /* Space them evenly */
align-items: center; /* Align them vertically */
gap: .5rem
}

.container {
width: 90%;
margin: 0 auto;
}

.text-center {
text-align: center;
}

.score-amount {
margin: 0;
margin-top: 10px;
}

.score-percentage {
font-size: 12px;
font-weight: semi-bold;
}

</style>

# Rapidata Imagen 4 偏好数据集

<a href="https://www.rapidata.ai">
<img src="https://cdn-uploads.huggingface.co/production/uploads/66f5624c42b853e73e0738eb/jfxR79bOztqaC6_yNNnGU.jpeg" width="400" alt="数据集可视化">
</a>

本**文本生成图像（Text-to-Image, T2I）**数据集包含来自7万余名独立标注者的超19.5万条人类标注反馈，仅用约1天时间通过[Rapidata Python应用程序接口（API）](https://docs.rapidata.ai)完成采集，全平台开放访问，非常适合大规模模型评估。
本次评估针对Imagen 4（imagen-4.0-ultra-generate-exp-05-20），从偏好性、连贯性与对齐性三个维度展开。

可前往我们的[官网](https://www.rapidata.ai/benchmark)查看最新的模型排名榜单。

如果您从本数据集获益并希望后续获取更多同类资源，欢迎为其点赞❤️

## 数据集概览
本文本生成图像（T2I）数据集包含来自7万余名独立标注者的超19.5万条人类标注反馈，采集周期仅约1天。本次评估针对Imagen 4（imagen-4.0-ultra-generate-exp-05-20），从偏好性、连贯性与对齐性三个维度展开。

本次评估采用1v1对比形式，将Imagen 4（imagen-4.0-ultra-generate-exp-05-20）与其余15款模型进行比对，分别为：Hidream I1 full、Halfmoon-4-4-2025、OpenAI 4o-26-3-25、Ideogram V2、Recraft V2、Lumina-15-2-25、Frames-23-1-25、Imagen-3、Flux-1.1-pro、Flux-1-pro、DALL-E 3、Midjourney-5.2、Stable Diffusion 3、Aurora以及Janus-7b。
> **注意：** 模型名称后的数字（如Halfmoon-4-4-2025）代表该模型生成对应图像的日期（2025年4月4日），用于明确所使用的模型版本信息。

## 对齐性
对齐性评分用于量化生成图像与输入提示文本的匹配程度。标注者被要求回答：“哪张图像更贴合描述文本？”

<div class="vertical-container">
<div class="container">
<div class="text-center">
<q>一只蓝色杯子与一部绿色手机。</q>
</div>
<div class="image-container">
<div>
<h3 class="score-amount">Imagen 4 </h3>
<div class="score-percentage">得分：100%</div>
<img style="border: 3px solid #18c54f;" src="https://cdn-uploads.huggingface.co/production/uploads/664dcc6296d813a7e15e170e/8JcDYOeviHCwVQo0vLMB-.jpeg" width=500>
</div>
<div>
<h3 class="score-amount">Frames-23-1-25 </h3>
<div class="score-percentage">得分：0%</div>
<img src="https://cdn-uploads.huggingface.co/production/uploads/664dcc6296d813a7e15e170e/WmHuNZY4M_NWUMSTSofNL.jpeg" width=500>
</div>
</div>
</div>

<div class="container">
<div class="text-center">
<q>一名手持旅行指南的游客正在历史遗迹中游览。</q>
</div>
<div class="image-container">
<div>
<h3 class="score-amount">Imagen 4</h3>
<div class="score-percentage">得分：0%</div>
<img src="https://cdn-uploads.huggingface.co/production/uploads/664dcc6296d813a7e15e170e/RiyfAra-C5QcCT9lDiohg.jpeg" width=500>
</div>
<div>
<h3 class="score-amount">Aurora</h3>
<div class="score-percentage">得分：100%</div>
<img style="border: 3px solid #18c54f;" src="https://cdn-uploads.huggingface.co/production/uploads/664dcc6296d813a7e15e170e/SPbiu_SE8S4vRv6p0OZHo.jpeg" width=500>
</div>
</div>
</div>
</div>

## 连贯性
连贯性评分用于衡量生成图像的逻辑一致性，以及是否存在视觉伪影或视觉瑕疵。在不查看原始提示文本的前提下，标注者被要求回答：“哪张图像的瑕疵更多，且更像是AI生成的？”

<div class="vertical-container">
<div class="container">
<div class="image-container">
<div>
<h3 class="score-amount">Imagen 4 </h3>
<div class="score-percentage">瑕疵评级：0%</div>
<img style="border: 3px solid #18c54f;" src="https://cdn-uploads.huggingface.co/production/uploads/664dcc6296d813a7e15e170e/X2jimr0Icoqb4RlX6jG1j.jpeg" width=500>
</div>
<div>
<h3 class="score-amount">Stable Diffusion 3 </h3>
<div class="score-percentage">瑕疵评级：100%</div>
<img src="https://cdn-uploads.huggingface.co/production/uploads/664dcc6296d813a7e15e170e/UUllaDc48oxLXxeXbF_CD.jpeg" width=500>
</div>
</div>
</div>

<div class="container">
<div class="image-container">
<div>
<h3 class="score-amount">Imagen 4 </h3>
<div class="score-percentage">瑕疵评级：100%</div>
<img src="https://cdn-uploads.huggingface.co/production/uploads/664dcc6296d813a7e15e170e/1a5dQzX2XxTLIUcgimocy.jpeg" width=500>
</div>
<div>
<h3 class="score-amount">Ideogram</h3>
<div class="score-percentage">瑕疵评级：0%</div>
<img style="border: 3px solid #18c54f;" src="https://cdn-uploads.huggingface.co/production/uploads/664dcc6296d813a7e15e170e/pI5B7RxysOcUhVEzdqHFq.jpeg" width=500>
</div>
</div>
</div>
</div>

## 偏好性
偏好性评分反映参与者对每张图像的视觉喜好程度，与提示文本无关。标注者被要求回答：“您更偏好哪张图像？”

<div class="vertical-container">
<div class="container">
<div class="image-container">
<div>
<h3 class="score-amount">Imagen 4</h3>
<div class="score-percentage">得分：81.83%</div>
<img style="border: 3px solid #18c54f;" src="https://cdn-uploads.huggingface.co/production/uploads/664dcc6296d813a7e15e170e/b3ny4C5fRdfnoIlK3ocFi.jpeg" width=500>
</div>
<div>
<h3 class="score-amount">Frames-23-1-25</h3>
<div class="score-percentage">得分：18.17%</div>
<img src="https://cdn-uploads.huggingface.co/production/uploads/664dcc6296d813a7e15e170e/KDJSo6s6FMKpXiovP03cJ.jpeg" width=500>
</div>
</div>
</div>

<div class="container">
<div class="image-container">
<div>
<h3 class="score-amount">Imagen 4 </h3>
<div class="score-percentage">得分：27.42%</div>
<img src="https://cdn-uploads.huggingface.co/production/uploads/664dcc6296d813a7e15e170e/-g7n1pP8QRU0Gcv-iKF4q.jpeg" width=500>
</div>
<div>
<h3 class="score-amount">Frames-23-1-25 </h3>
<div class="score-percentage">得分：72.58%</div>
<img style="border: 3px solid #18c54f;" src="https://cdn-uploads.huggingface.co/production/uploads/664dcc6296d813a7e15e170e/BMXzpF1loC3wVfWuyj5VX.jpeg" width=500>
</div>
</div>
</div>
</div>

## 关于Rapidata
Rapidata的技术使大规模人类反馈采集比以往任何时候都更加快速、便捷。请访问[rapidata.ai](https://www.rapidata.ai/)了解更多关于我们如何革新AI开发领域的人类反馈采集技术的信息。