flux1.1-likert-scale-preference

# Flux1.1 Likert Scale Text-to-Image Alignment Evaluation

This dataset contains images generated using Flux1.1 [pro] based on the prompts from [our text-to-image generation benchmark](https://rapidata.ai/blog/genai-blog-post).
Where the benchmark generally focuses on pairwise comparisons to rank different image generation models against each other, this Likert-scale dataset focuses on one
particular model and aims to reveal the particular nuances and highlight strong and weaks points of the model.

If you get value from this dataset and would like to see more in the future, please consider liking it.

## Dataset Details
Annotators were presented with an image and a prompt and asked to rate how well the image matched the prompt with options on a scale of 1-5: `1: Not at all`, `2: A little`,
`3: Moderately`, `4: Very well`, and `5: Perfectly`. For each image, at least 30 responses have been collected, for a total of ~35.5k responses. The `score` reported is a
weighted average of the responses.

The images, prompts, and responses are available through the parquet file, however they can also be downloaded directly via the .csv and .zip files. We additionally provide
the 'raw' responses as a .json which contains additional metadata for each individual response.

## Usage
The easiest way to use the dataset is through the Huggingface datasets package:
```python
from datasets import load_dataset

ds = load_dataset("Rapidata/flux1.1-likert-scale-preference")
```

## Collecting Custom Dataset
The responses for this dataset were collected in a few hours using Rapidata's network of annotators, which can be easily utilised through Rapidata's API. If you are interested
in creating a similar dataset for a different model, check out [our API documentation](https://rapidataai.github.io/rapidata-python-sdk/).

# Flux1.1李克特量表式文本-图像对齐评估数据集
本数据集包含基于[我们的文本-图像生成基准测试](https://rapidata.ai/blog/genai-blog-post)中的提示词，使用Flux1.1专业版生成的图像。与该基准测试通常聚焦于通过两两对比对不同图像生成模型进行排序不同，本李克特量表数据集仅针对单一模型，旨在揭示该模型的特定细节差异，并凸显其优劣之处。
若您从本数据集获益并希望未来获取更多同类资源，不妨为该数据集点赞。

## 数据集详情
标注人员会收到一张图像与对应的提示词，并需按照1-5分量表对图像与提示词的匹配程度进行评分，评分选项依次为：`1: 完全不匹配`，`2: 略有匹配`，`3: 中等匹配`，`4: 高度匹配`，`5: 完美匹配`。每张图像至少收集了30份标注结果，整体累计约3.55万份响应。报告中的`score`为所有标注结果的加权平均值。
图像、提示词及标注结果可通过parquet文件获取，也可直接通过.csv与.zip文件下载。此外，我们还提供了包含每份独立标注结果额外元数据的原始响应.json文件。

## 使用方法
使用本数据集最简便的方式是通过Hugging Face datasets库：
python
from datasets import load_dataset

ds = load_dataset("Rapidata/flux1.1-likert-scale-preference")

## 自定义数据集采集
本数据集的标注结果通过Rapidata的标注人员网络在数小时内完成采集，可通过Rapidata的API轻松调用该服务。若您希望为其他模型构建同类数据集，请查阅[我们的API文档](https://rapidataai.github.io/rapidata-python-sdk/).