Imagen-4-ultra-24-7-25

<style>

.vertical-container {
display: flex;
flex-direction: column;
gap: 60px;
}

.horizontal-container {
display: flex;
flex-direction: row;
justify-content: center;
gap: 60px;
}

.image-container img {
max-height: 250px; /* Set the desired height */
margin:0;
object-fit: contain; /* Ensures the aspect ratio is maintained */
width: auto; /* Adjust width automatically based on height */
box-sizing: content-box;
}

.image-container img.big {
max-height: 350px; /* Set the desired height */
}

.image-container {
display: flex; /* Aligns images side by side */
justify-content: space-around; /* Space them evenly */
align-items: center; /* Align them vertically */
gap: .5rem
}

.container {
width: 90%;
margin: 0 auto;
}

.text-center {
text-align: center;
}

.score-amount {
margin: 0;
margin-top: 10px;
}

.score-percentage {Score:
font-size: 12px;
font-weight: semi-bold;
}

</style>

# Rapidata Imagen 4 Ultra 24.7.25 Preference

<a href="https://www.rapidata.ai">
<img src="https://cdn-uploads.huggingface.co/production/uploads/66f5624c42b853e73e0738eb/jfxR79bOztqaC6_yNNnGU.jpeg" width="400" alt="Dataset visualization">
</a>

This T2I dataset contains over ~400'000 human responses from over ~83'000 individual annotators, collected in less than 7h using the [Rapidata Python API](https://docs.rapidata.ai), accessible to anyone and ideal for large scale evaluation.
Evaluating Imagen 4 Ultra (version from 24.7.2025) across three categories: preference, coherence, and alignment.

Explore our latest model rankings on our [website](https://www.rapidata.ai/benchmark).

If you get value from this dataset and would like to see more in the future, please consider liking it ❤️

To add your own model to the benchmark send us an e-mail at: [e-mail]

## Overview

The evaluation consists of 1v1 comparisons between 4 Ultra (version from 24.7.2025) and 17 other models:
- 4o
- Flux-1-pro
- Flux-1.1-pro
- imagen 4 ultra
- Aurora
- Imagen-3
- DALL-E 3
- Midjourney-5.2
- Frames-23-1-25
- Stable Diffusion 3
- Janus-7b.
- hidream-l1-full
- Recraft V2
- Ideogram V2
- halfmoon-4-4-25
- Lumina-15-2-25
- Imagen 4 Ultra 20.5.25

## Alignment

The alignment score quantifies how well an video matches its prompt. Users were asked: "Which image matches the description better?".

<div class="vertical-container">
<div class="container">
<div class="text-center">
<q>A device consisting of a circular canopy of cloth on a folding metal frame supported by a central rod, used as protection against rain or sometimes sun.</q>
</div>
<div class="image-container">
<div>
<h3 class="score-amount">Imagen 4 Ultra 24.7.25 </h3>
<div class="score-percentage">Score: 100%</div>
<img style="border: 5px solid #18c54f;" src="https://cdn-uploads.huggingface.co/production/uploads/672b7d79fd1e92e3c3567435/1rDESx99UMCQckIGLPuR5.jpeg" width=500>
</div>
<div>
<h3 class="score-amount">Stable Diffusion 3 </h3>
<div class="score-percentage">Score: 0%</div>
<img src="https://cdn-uploads.huggingface.co/production/uploads/672b7d79fd1e92e3c3567435/ggn_hbMMwin60_K7X4mVf.jpeg" width=500>
</div>
</div>
</div>

<div class="container">
<div class="text-center">
<q>a fluffy pillow and a leather belt</q>
</div>
<div class="image-container">
<div>
<h3 class="score-amount">Imagen 4 Ultra 24.7.25 </h3>
<div class="score-percentage">Score: 0%</div>
<img src="https://cdn-uploads.huggingface.co/production/uploads/672b7d79fd1e92e3c3567435/0fepNGUb1dpyd7VwXno-u.jpeg" width=500>
</div>
<div>
<h3 class="score-amount">Flux 1 Pro </h3>
<div class="score-percentage">Score: 100%</div>
<img style="border: 5px solid #18c54f;" src="https://cdn-uploads.huggingface.co/production/uploads/672b7d79fd1e92e3c3567435/oRg-y4McFQrbueq7OPsjW.jpeg" width=500>
</div>
</div>
</div>
</div>

## Coherence

The coherence score measures whether the generated video is logically consistent and free from artifacts or visual glitches. Without seeing the original prompt, users were asked: "Which image has **more** glitches and is **more** likely to be AI generated?"

<div class="vertical-container">
<div class="container">
<div class="image-container">
<div>
<h3 class="score-amount">Imagen 4 Ultra 24.7.25 </h3>
<div class="score-percentage">Glitch Rating: 0%</div>
<img style="border: 5px solid #18c54f;" src="https://cdn-uploads.huggingface.co/production/uploads/672b7d79fd1e92e3c3567435/9pi5G1y_lTbt9THnOQFNi.jpeg" width=500>
</div>
<div>
<h3 class="score-amount">Janus 7b </h3>
<div class="score-percentage">Glitch Rating: 100%</div>
<img src="https://cdn-uploads.huggingface.co/production/uploads/672b7d79fd1e92e3c3567435/TcaVx3rRYpVsEUAX7AzoE.jpeg" width=500>
</div>
</div>
</div>

<div class="container">
<div class="image-container">
<div>
<h3 class="score-amount">Imagen 4 Ultra 24.7.25 </h3>
<div class="score-percentage">Glitch Rating: 100%</div>
<img src="https://cdn-uploads.huggingface.co/production/uploads/672b7d79fd1e92e3c3567435/oVDb4Xz4TagKuIs_DExFG.jpeg" width=500>
</div>
<div>
<h3 class="score-amount">Aurora </h3>
<div class="score-percentage">Glitch Rating: 0%</div>
<img style="border: 5px solid #18c54f;" src="https://cdn-uploads.huggingface.co/production/uploads/672b7d79fd1e92e3c3567435/VDOxnq8SeQyUUcwk5Zl9U.jpeg" width=500>
</div>
</div>
</div>
</div>

## Preference

The preference score reflects how visually appealing participants found each image, independent of the prompt. Users were asked: "Which image do you prefer?"

<div class="vertical-container">
<div class="container">
<div class="image-container">
<div>
<h3 class="score-amount">Imagen 4 Ultra 24.7.25 </h3>
<div class="score-percentage">Score: 100%</div>
<img style="border: 5px solid #18c54f;" src="https://cdn-uploads.huggingface.co/production/uploads/672b7d79fd1e92e3c3567435/gRNaZ0wy7NYHYeoZUFph1.jpeg" width=500>
</div>
<div>
<h3 class="score-amount">Lumina-15-2-25</h3>
<div class="score-percentage">Score: 0%</div>
<img src="https://cdn-uploads.huggingface.co/production/uploads/672b7d79fd1e92e3c3567435/a8ySA_UpQCGz3_K51ImSz.jpeg" width=500>
</div>
</div>
</div>

<div class="container">
<div class="image-container">
<div>
<h3 class="score-amount">Imagen 4 Ultra 24.7.25 </h3>
<div class="score-percentage">Score: 0%</div>
<img src="https://cdn-uploads.huggingface.co/production/uploads/672b7d79fd1e92e3c3567435/fRKhM8s_1W03MCc3ZCNbH.jpeg" width=500>
</div>
<div>
<h3 class="score-amount">Hidream </h3>
<div class="score-percentage">Score: 100%</div>
<img style="border: 5px solid #18c54f;" src="https://cdn-uploads.huggingface.co/production/uploads/672b7d79fd1e92e3c3567435/hgqqrRz1qe3KYr97f_ovr.jpeg" width=500>
</div>
</div>
</div>
</div>

## About Rapidata

Rapidata's technology makes collecting human feedback at scale faster and more accessible than ever before. Visit [rapidata.ai](https://www.rapidata.ai/) to learn more about how we're revolutionizing human feedback collection for AI development.

# Rapidata Imagen 4 Ultra 24.7.25 偏好性评测数据集

<a href="https://www.rapidata.ai"><img src="https://cdn-uploads.huggingface.co/production/uploads/66f5624c42b853e73e0738eb/jfxR79bOztqaC6_yNNnGU.jpeg" width="400" alt="数据集可视化"></a>

本文本到图像（Text-to-Image, T2I）数据集收录了来自超8.3万名独立标注者的约40万条人类标注反馈，通过[Rapidata Python API](https://docs.rapidata.ai)在7小时内完成采集，全量开放且适配大规模评测场景。

本次评测针对Imagen 4 Ultra（2025年7月24日版本）从三个维度展开：偏好性、连贯性与对齐性。

可访问我们的[官方网站](https://www.rapidata.ai/benchmark)查看最新的模型排名榜单。

若本数据集对您的研究有所助益并希望后续获取更多同类资源，欢迎点赞支持❤️

若希望将您的模型加入评测基准，请发送邮件至：[e-mail]

## 概述

本次评测采用1v1对比形式，将Imagen 4 Ultra（2025年7月24日版本）与其余17款模型进行比对，具体包括：
- 4o
- Flux-1-pro
- Flux-1.1-pro
- Imagen 4 Ultra
- Aurora
- Imagen-3
- DALL-E 3
- Midjourney-5.2
- Frames-23-1-25
- Stable Diffusion 3
- Janus-7b
- hidream-l1-full
- Recraft V2
- Ideogram V2
- halfmoon-4-4-25
- Lumina-15-2-25
- Imagen 4 Ultra 20.5.25

## 对齐性维度

对齐性评分用于量化生成图像与输入提示词的匹配程度。标注任务问题为：「哪张图像更贴合描述内容？」

<div class="vertical-container">
<div class="container">
<div class="text-center">
<q>一种由中央支架支撑的折叠金属框架搭配圆形布质顶篷组成的装置，用于防雨或遮阳</q>
</div>
<div class="image-container">
<div>
<h3 class="score-amount">Imagen 4 Ultra 24.7.25</h3>
<div class="score-percentage">得分：100%</div>
<img style="border: 5px solid #18c54f;" src="https://cdn-uploads.huggingface.co/production/uploads/672b7d79fd1e92e3c3567435/1rDESx99UMCQckIGLPuR5.jpeg" width=500>
</div>
<div>
<h3 class="score-amount">Stable Diffusion 3</h3>
<div class="score-percentage">得分：0%</div>
<img src="https://cdn-uploads.huggingface.co/production/uploads/672b7d79fd1e92e3c3567435/ggn_hbMMwin60_K7X4mVf.jpeg" width=500>
</div>
</div>
</div>

<div class="container">
<div class="text-center">
<q>一个蓬松的枕头与一条皮质腰带</q>
</div>
<div class="image-container">
<div>
<h3 class="score-amount">Imagen 4 Ultra 24.7.25</h3>
<div class="score-percentage">得分：0%</div>
<img src="https://cdn-uploads.huggingface.co/production/uploads/672b7d79fd1e92e3c3567435/0fepNGUb1dpyd7VwXno-u.jpeg" width=500>
</div>
<div>
<h3 class="score-amount">Flux 1 Pro</h3>
<div class="score-percentage">得分：100%</div>
<img style="border: 5px solid #18c54f;" src="https://cdn-uploads.huggingface.co/production/uploads/672b7d79fd1e92e3c3567435/oRg-y4McFQrbueq7OPsjW.jpeg" width=500>
</div>
</div>
</div>
</div>

## 连贯性维度

连贯性评分用于衡量生成图像的逻辑自洽性，以及是否存在人工痕迹或视觉瑕疵。在不查看原始提示词的前提下，标注任务问题为：「哪张图像的瑕疵更多，且更像是AI生成内容？」

<div class="vertical-container">
<div class="container">
<div class="image-container">
<div>
<h3 class="score-amount">Imagen 4 Ultra 24.7.25</h3>
<div class="score-percentage">瑕疵率：0%</div>
<img style="border: 5px solid #18c54f;" src="https://cdn-uploads.huggingface.co/production/uploads/672b7d79fd1e92e3c3567435/9pi5G1y_lTbt9THnOQFNi.jpeg" width=500>
</div>
<div>
<h3 class="score-amount">Janus 7b</h3>
<div class="score-percentage">瑕疵率：100%</div>
<img src="https://cdn-uploads.huggingface.co/production/uploads/672b7d79fd1e92e3c3567435/TcaVx3rRYpVsEUAX7AzoE.jpeg" width=500>
</div>
</div>
</div>

<div class="container">
<div class="image-container">
<div>
<h3 class="score-amount">Imagen 4 Ultra 24.7.25</h3>
<div class="score-percentage">瑕疵率：100%</div>
<img src="https://cdn-uploads.huggingface.co/production/uploads/672b7d79fd1e92e3c3567435/oVDb4Xz4TagKuIs_DExFG.jpeg" width=500>
</div>
<div>
<h3 class="score-amount">Aurora</h3>
<div class="score-percentage">瑕疵率：0%</div>
<img style="border: 5px solid #18c54f;" src="https://cdn-uploads.huggingface.co/production/uploads/672b7d79fd1e92e3c3567435/VDOxnq8SeQyUUcwk5Zl9U.jpeg" width=500>
</div>
</div>
</div>
</div>

## 偏好性维度

偏好性评分用于反映参与者对图像视觉效果的主观喜好程度，与原始提示词无关。标注任务问题为：「你更偏好哪张图像？」

<div class="vertical-container">
<div class="container">
<div class="image-container">
<div>
<h3 class="score-amount">Imagen 4 Ultra 24.7.25</h3>
<div class="score-percentage">得分：100%</div>
<img style="border: 5px solid #18c54f;" src="https://cdn-uploads.huggingface.co/production/uploads/672b7d79fd1e92e3c3567435/gRNaZ0wy7NYHYeoZUFph1.jpeg" width=500>
</div>
<div>
<h3 class="score-amount">Lumina-15-2-25</h3>
<div class="score-percentage">得分：0%</div>
<img src="https://cdn-uploads.huggingface.co/production/uploads/672b7d79fd1e92e3c3567435/a8ySA_UpQCGz3_K51ImSz.jpeg" width=500>
</div>
</div>
</div>

<div class="container">
<div class="image-container">
<div>
<h3 class="score-amount">Imagen 4 Ultra 24.7.25</h3>
<div class="score-percentage">得分：0%</div>
<img src="https://cdn-uploads.huggingface.co/production/uploads/672b7d79fd1e92e3c3567435/fRKhM8s_1W03MCc3ZCNbH.jpeg" width=500>
</div>
<div>
<h3 class="score-amount">Hidream</h3>
<div class="score-percentage">得分：100%</div>
<img style="border: 5px solid #18c54f;" src="https://cdn-uploads.huggingface.co/production/uploads/672b7d79fd1e92e3c3567435/hgqqrRz1qe3KYr97f_ovr.jpeg" width=500>
</div>
</div>
</div>
</div>

## 关于Rapidata

Rapidata的技术让大规模人类反馈采集比以往任何时候都更快捷、更普惠。访问[rapidata.ai](https://www.rapidata.ai/)了解更多我们如何革新AI开发领域的人类反馈采集技术。

搜集汇总

背景与挑战

背景概述

该数据集是一个文本到图像（T2I）评估数据集，包含约40万条人类响应，由超过8.3万名注释者通过Rapidata Python API在7小时内收集。它用于评估Imagen 4 Ultra（版本24.7.2025）在偏好、一致性和对齐性方面的性能，并与其他17个模型进行了比较。