open-image-preferences-v1-more-results

<a href="https://www.rapidata.ai">
<img src="https://cdn-uploads.huggingface.co/production/uploads/66f5624c42b853e73e0738eb/jfxR79bOztqaC6_yNNnGU.jpeg" width="250" alt="Rapidata Logo">
</a>

We wanted to contribute to the challenge posed by the data-is-better-together community (description below). We collected 170'000 preferences using our API from people all around the world in rougly 3 days (docs.rapidata.ai):

If you get value from this dataset and would like to see more in the future, please consider liking it.

# Dataset Card for image-preferences-results [Original](https://huggingface.co/datasets/data-is-better-together/open-image-preferences-v1)

<style>
.row {
display: flex;
justify-content: space-between;
width: 100%;
}
#container {
display: flex;
flex-direction: column;
font-family: Arial, sans-serif;
width: 98%
}
.prompt {
margin-bottom: 10px;
font-size: 16px;
line-height: 1.4;
color: #333;
background-color: #f8f8f8;
padding: 10px;
border-radius: 5px;
box-shadow: 0 1px 3px rgba(0,0,0,0.1);
}
.image-container {
display: flex;
gap: 10px;
}
.column {
flex: 1;
position: relative;
}
img {
max-width: 100%;
height: auto;
display: block;
}
.image-label {
position: absolute;
top: 10px;
right: 10px;
background-color: rgba(255, 255, 255, 0.7);
color: black;
padding: 5px 10px;
border-radius: 5px;
font-weight: bold;
}
</style>

<div class="row">
<div class="column">
<div id="container">
<div class="prompt"><strong>Prompt:</strong> Anime-style concept art of a Mayan Quetzalcoatl biomutant, dystopian world, vibrant colors, 4K.</div>
<div class="image-container">
<div class="column">
<img src="https://huggingface.co/datasets/data-is-better-together/open-image-preferences-v1/resolve/main/image_simplified_sd/1258.jpg">
<div class="image-label">Image 1</div>
</div>
<div class="column">
<img src="https://huggingface.co/datasets/data-is-better-together/open-image-preferences-v1/resolve/main/image_simplified_dev/1258.jpg">
<div class="image-label">Image 2</div>
</div>
</div>
</div>
</div>
<div class="column">

<div id="container">
<div class="prompt"><strong>Prompt:</strong> 8-bit pixel art of a blue knight, green car, and glacier landscape in Norway, fantasy style, colorful and detailed.</div>
<div class="image-container">
<div class="column">
<img src="https://huggingface.co/datasets/data-is-better-together/open-image-preferences-v1/resolve/main/image_simplified_dev/1210.jpg">
<div class="image-label">Image 1</div>
</div>
<div class="column">
<img src="https://huggingface.co/datasets/data-is-better-together/open-image-preferences-v1/resolve/main/image_simplified_sd/1210.jpg">
<div class="image-label">Image 2</div>
</div>
</div>
</div>
</div>
</div>

- **Goal**: This project aims to create 10K text-to-image preference pairs. These pairs can be used to evaluate the performance of image generation models across a wide variety of common image categories, based on prompt with varying levels of difficulty.
- **How**: We use the prompts from [fal/imgsys-results](https://huggingface.co/datasets/fal/imgsys-results), these prompts are evolved based on complexity and quality for various image categories. We then asked the community to annotate the preference between two generated images for each prompt.
- **Result**: Rapidata collected over 170'000 individual preferences from people all around the globe. There were 17k image pairs for each of them we collected roughly 10 preference annotations.
- **Methodology**: Annotators were asked "Which image do you prefer based on the description?". They were given the option to choose between the two images, the prompt was also displayed. "Both" was not given as an option. Each pair was shown to 10 annoatators, the positions of the images were shuffled at random. Each Annotator has a trust worthyness score attached to each annotation, which can be found in the detailed results in the dataset.
- **Format**: The annotations are given once in the same format as the original smaller result set and once in a more detailed format that follows the usual Rapidata format, present in our other preference datasets.
# More Open Preference Datasets:
We tried to stay as close as possible to the format proposed by the data-is-better-together community. This has the limitation that the quality of the style, coherence, and alignment are melted into one score. We have collected multiple datasets where we differentiate between these modalities:
- https://huggingface.co/collections/Rapidata/flux-sd3-mj-dalle-human-annotation-sets-675ae83c8ad7551e497e2c29
- https://huggingface.co/collections/Rapidata/rapidata-benchmark-data-675ae93f0193e1f84d66d083

If you have any questions, feel free to reach out to me at [e-mail]

<a href="https://www.rapidata.ai"><img src="https://cdn-uploads.huggingface.co/production/uploads/66f5624c42b853e73e0738eb/jfxR79bOztqaC6_yNNnGU.jpeg" width="250" alt="Rapidata标志"></a>

我们希望为「数据共生（data-is-better-together）」社区提出的挑战贡献力量（社区说明如下）。我们通过自有API，在约3天内从全球各地的用户处收集了170,000条偏好标注（相关文档：docs.rapidata.ai）：

若您从本数据集获益并希望未来获得更多同类资源，欢迎为该数据集点赞。

# 图像偏好结果数据集卡片 [原始数据集](https://huggingface.co/datasets/data-is-better-together/open-image-preferences-v1)

<style>
.row {
display: flex;
justify-content: space-between;
width: 100%;
}
#container {
display: flex;
flex-direction: column;
font-family: Arial, sans-serif;
width: 98%
}
.prompt {
margin-bottom: 10px;
font-size: 16px;
line-height: 1.4;
color: #333;
background-color: #f8f8f8;
padding: 10px;
border-radius: 5px;
box-shadow: 0 1px 3px rgba(0,0,0,0.1);
}
.image-container {
display: flex;
gap: 10px;
}
.column {
flex: 1;
position: relative;
}
img {
max-width: 100%;
height: auto;
display: block;
}
.image-label {
position: absolute;
top: 10px;
right: 10px;
background-color: rgba(255, 255, 255, 0.7);
color: black;
padding: 5px 10px;
border-radius: 5px;
font-weight: bold;
}
</style>

<div class="row">
<div class="column">
<div id="container">
<div class="prompt"><strong>提示词：</strong> 玛雅羽蛇神生物改造体的动画风格概念艺术，反乌托邦世界，色彩鲜艳，4K分辨率。</div>
<div class="image-container">
<div class="column">
<img src="https://huggingface.co/datasets/data-is-better-together/open-image-preferences-v1/resolve/main/image_simplified_sd/1258.jpg">
<div class="image-label">图像1</div>
</div>
<div class="column">
<img src="https://huggingface.co/datasets/data-is-better-together/open-image-preferences-v1/resolve/main/image_simplified_dev/1258.jpg">
<div class="image-label">图像2</div>
</div>
</div>
</div>
</div>
<div class="column">

<div id="container">
<div class="prompt"><strong>提示词：</strong> 8位像素艺术风格的蓝色骑士、绿色汽车与挪威冰川景观，奇幻风格，色彩丰富且细节饱满。</div>
<div class="image-container">
<div class="column">
<img src="https://huggingface.co/datasets/data-is-better-together/open-image-preferences-v1/resolve/main/image_simplified_dev/1210.jpg">
<div class="image-label">图像1</div>
</div>
<div class="column">
<img src="https://huggingface.co/datasets/data-is-better-together/open-image-preferences-v1/resolve/main/image_simplified_sd/1210.jpg">
<div class="image-label">图像2</div>
</div>
</div>
</div>
</div>
</div>

- **项目目标**：本项目旨在构建10,000组文本到图像（text-to-image）偏好配对样本。这些样本可用于基于不同难度等级的提示词，评估各类常见图像类别下的图像生成模型性能。
- **实现路径**：我们采用[fal/imgsys-results](https://huggingface.co/datasets/fal/imgsys-results)中的提示词，这些提示词针对各类图像类别基于复杂度与质量进行了优化升级。随后我们邀请社区用户为每个提示词对应的两张生成图像标注偏好。
- **项目成果**：Rapidata从全球用户处累计收集了超过170,000条独立偏好标注。本数据集共包含17,000组图像配对，每组配对均收集了约10条偏好标注。
- **标注流程**：标注者会被问及「基于上述描述，你更偏好哪一张图像？」，系统会同时展示提示词与两张图像，并仅提供两个选项供其选择（无「两者都偏好」选项）。每组图像配对会被随机展示给10位标注者，且图像的展示位置会随机打乱。每位标注者的每一条标注均附带可信度评分，相关细节可在数据集的详细结果中查看。
- **数据格式**：本数据集的标注文件提供两种格式：一种与原始小型结果集格式一致，另一种则采用Rapidata标准的详细格式（该格式也应用于我们的其他偏好数据集）。

# 更多开放偏好数据集：
我们尽可能遵循「数据共生（data-is-better-together）」社区提出的数据集格式，但该格式存在一定局限：其将风格质量、内容连贯性与图像对齐度合并为单一评分。我们还收集了多组可区分上述维度的数据集，具体如下：
- https://huggingface.co/collections/Rapidata/flux-sd3-mj-dalle-human-annotation-sets-675ae83c8ad7551e497e2c29
- https://huggingface.co/collections/Rapidata/rapidata-benchmark-data-675ae93f0193e1f84d66d083

如有任何疑问，欢迎通过[e-mail]联系我们。