q-future/q-eval-plus

---
dataset_info:
default:
description: ""
citation: ""
homepage: ""
license: cc-by-4.0
features:
- name: data
type: string
splits:
- name: train
num_bytes: 0
num_examples: 0
- name: test
num_bytes: 0
num_examples: 0
image_quality:
description: "Image quality assessment dataset"
citation: ""
homepage: ""
license: cc-by-4.0
data_files:
- path: info/image_quality_pairs_train.json
split: train
- path: info/image_quality_pairs_test.json
split: test
features:
- name: data
type: string
image_alignment:
description: "Image-text alignment assessment dataset"
citation: ""
homepage: ""
license: cc-by-4.0
data_files:
- path: info/image_alignment_pairs_train.json
split: train
- path: info/image_alignment_pairs_test.json
split: test
features:
- name: data
type: string
video_quality:
description: "Video quality assessment dataset"
citation: ""
homepage: ""
license: cc-by-4.0
data_files:
- path: info/video_quality_pairs_train.json
split: train
- path: info/video_quality_pairs_test.json
split: test
features:
- name: data
type: string
video_alignment:
description: "Video-text alignment assessment dataset"
citation: ""
homepage: ""
license: cc-by-4.0
data_files:
- path: info/video_alignment_pairs_train.json
split: train
- path: info/video_alignment_pairs_test.json
split: test
features:
- name: data
type: string
---

# Q-Eval Plus Dataset

## Overview

The Q-Eval Plus dataset is a comprehensive benchmark for evaluating image and video generation models. It contains paired datasets across four evaluation tasks with both training and test splits.

## Dataset Structure

The dataset includes the following evaluation tasks:

1. **Image Quality** - Assesses the quality of generated images
2. **Image Alignment** - Evaluates image-text alignment quality
3. **Video Quality** - Assesses the quality of generated videos
4. **Video Alignment** - Evaluates video-text alignment quality

Each task has:
- Training set (`*_train.json`)
- Test set (`*_test.json`)

### Dataset Configurations

Use the `name` parameter to select which task to download:

| Configuration | Description | Files |
|---------------|-------------|-------|
| `image_quality` | Image quality assessment | `image_quality_pairs_train.json`, `image_quality_pairs_test.json` |
| `image_alignment` | Image-text alignment | `image_alignment_pairs_train.json`, `image_alignment_pairs_test.json` |
| `video_quality` | Video quality assessment | `video_quality_pairs_train.json`, `video_quality_pairs_test.json` |
| `video_alignment` | Video-text alignment | `video_alignment_pairs_train.json`, `video_alignment_pairs_test.json` |

## Download Methods

### Method 1: Using Hugging Face `datasets` Library (Recommended)

The easiest way to download the dataset:

```python
from datasets import load_dataset

# Download image_quality training set
train_dataset = load_dataset("q-future/q-eval-plus",
name="image_quality",
split="train")

# Download image_quality test set
test_dataset = load_dataset("q-future/q-eval-plus",
name="image_quality",
split="test")

# Access the data
for example in train_dataset:
print(example)
```

### Method 2: Download All Data by Task

Download all splits for a specific task using the `split` parameter:

```python
from datasets import load_dataset

# Image Quality - Training and Test
img_quality_train = load_dataset("q-future/q-eval-plus", name="image_quality", split="train")
img_quality_test = load_dataset("q-future/q-eval-plus", name="image_quality", split="test")

# Image Alignment - Training and Test
img_align_train = load_dataset("q-future/q-eval-plus", name="image_alignment", split="train")
img_align_test = load_dataset("q-future/q-eval-plus", name="image_alignment", split="test")

# Video Quality - Training and Test
vid_quality_train = load_dataset("q-future/q-eval-plus", name="video_quality", split="train")
vid_quality_test = load_dataset("q-future/q-eval-plus", name="video_quality", split="test")

# Video Alignment - Training and Test
vid_align_train = load_dataset("q-future/q-eval-plus", name="video_alignment", split="train")
vid_align_test = load_dataset("q-future/q-eval-plus", name="video_alignment", split="test")
```

### Method 3: Batch Download All Datasets

Download all datasets at once:

```python
from datasets import load_dataset

# Define all task configurations
tasks = [
("image_quality", "train"),
("image_quality", "test"),
("image_alignment", "train"),
("image_alignment", "test"),
("video_quality", "train"),
("video_quality", "test"),
("video_alignment", "train"),
("video_alignment", "test"),
]

datasets = {}
for task, split in tasks:
dataset = load_dataset("q-future/q-eval-plus", name=task, split=split)
datasets[f"{task}_{split}"] = dataset
print(f"Downloaded {task} ({split})")
```

### Method 4: Using Command Line

Download the dataset using the Hugging Face CLI:

```bash
# Install the Hugging Face Hub library if not already installed
pip install huggingface_hub

# Download all files to a local directory
huggingface-cli download q-future/q-eval-plus --repo-type dataset --local-dir ./q-eval-plus
```

## Dataset Format

Each dataset file is a JSON array containing evaluation items. Here's an example structure:

```json
[
[
{
"model": "sd3.0-medium",
"prompt": "A bakery window displaying a cake...",
"gt_score": 5.0,
"image_path": "Images/sd3.0-medium/image.png"
},
{
"model": "wanx-en",
"prompt": "A bakery window displaying a cake...",
"gt_score": 3.3,
"image_path": "Images/wanx-en/image.png"
},
{
"choices": [
{
"type": "single",
"question": "What is the primary reason...",
"options": [...],
"answer": 0
}
]
}
]
]
```

### Field Descriptions

- **model**: The model used to generate the content
- **prompt**: The text prompt used for generation
- **gt_score**: Ground truth quality score
- **image_path** / **video_path**: Path to the generated content
- **choices**: Multiple choice questions for evaluation
- **type**: Type of question ("single" for single-choice)
- **question**: The evaluation question
- **options**: List of answer options
- **answer**: Index of the correct answer

## Installation Requirements

```bash
# Install required packages
pip install datasets
pip install huggingface_hub
```

## Statistics

| Task | Train Samples | Test Samples |
|------|--------------|-------------|
| Image Alignment | - | - |
| Image Quality | - | - |
| Video Alignment | - | - |
| Video Quality | - | - |

## Citation

If you use this dataset in your research, please cite:

```bibtex
@dataset{qeval_plus_2024,
title={Q-Eval Plus},
author={Q-Future},
year={xx},
url={https://huggingface.co/datasets/q-future/q-eval-plus}
}
```

## License

## Contact

For questions or issues, please visit the [dataset repository](https://huggingface.co/datasets/q-future/q-eval-plus).

## Related Links

- [Hugging Face Datasets Documentation](https://huggingface.co/docs/datasets/)
- [Q-Eval Plus GitHub Repository](https://github.com/Q-Future/Q-Eval-plus)
- [Model Evaluation Benchmark](https://huggingface.co/q-future)

dataset_info:
默认配置:
描述: ""
引用文献: ""
项目主页: ""
许可证: cc-by-4.0
特征:
- 名称: data
类型: 字符串
划分:
- 名称: train
字节数: 0
样本数: 0
- 名称: test
字节数: 0
样本数: 0
图像质量评估:
描述: "图像质量评估数据集"
引用文献: ""
项目主页: ""
许可证: cc-by-4.0
数据文件:
- 路径: info/image_quality_pairs_train.json
划分: train
- 路径: info/image_quality_pairs_test.json
划分: test
特征:
- 名称: data
类型: 字符串
图像-文本对齐评估:
描述: "图像-文本对齐评估数据集"
引用文献: ""
项目主页: ""
许可证: cc-by-4.0
数据文件:
- 路径: info/image_alignment_pairs_train.json
划分: train
- 路径: info/image_alignment_pairs_test.json
划分: test
特征:
- 名称: data
类型: 字符串
视频质量评估:
描述: "视频质量评估数据集"
引用文献: ""
项目主页: ""
许可证: cc-by-4.0
数据文件:
- 路径: info/video_quality_pairs_train.json
划分: train
- 路径: info/video_quality_pairs_test.json
划分: test
特征:
- 名称: data
类型: 字符串
视频-文本对齐评估:
描述: "视频-文本对齐评估数据集"
引用文献: ""
项目主页: ""
许可证: cc-by-4.0
数据文件:
- 路径: info/video_alignment_pairs_train.json
划分: train
- 路径: info/video_alignment_pairs_test.json
划分: test
特征:
- 名称: data
类型: 字符串

# Q-Eval Plus 数据集

## 概述
Q-Eval Plus 数据集是用于评估图像与视频生成模型的综合性基准测试集，涵盖四类评估任务的配对数据集，并包含训练集与测试集划分。

## 数据集结构
数据集包含以下四类评估任务：
1. **图像质量**：用于评估生成图像的质量
2. **图像-文本对齐**：用于评估图像与文本的对齐质量
3. **视频质量**：用于评估生成视频的质量
4. **视频-文本对齐**：用于评估视频与文本的对齐质量

每个任务均包含：
- 训练集（文件格式为`*_train.json`）
- 测试集（文件格式为`*_test.json`）

### 数据集配置
可通过`name`参数选择需下载的任务：

| 配置名称 | 任务描述 | 对应文件 |
|------------------|------------------------------|--------------------------------------------------------------------------|
| `image_quality` | 图像质量评估任务 | `image_quality_pairs_train.json`、`image_quality_pairs_test.json` |
| `image_alignment`| 图像-文本对齐评估任务 | `image_alignment_pairs_train.json`、`image_alignment_pairs_test.json` |
| `video_quality` | 视频质量评估任务 | `video_quality_pairs_train.json`、`video_quality_pairs_test.json` |
| `video_alignment`| 视频-文本对齐评估任务 | `video_alignment_pairs_train.json`、`video_alignment_pairs_test.json` |

## 下载方法
### 方法1：使用Hugging Face `datasets`库（推荐）
这是最便捷的数据集下载方式：
python
from datasets import load_dataset

# 下载图像质量评估任务的训练集
train_dataset = load_dataset("q-future/q-eval-plus",
name="image_quality",
split="train")

# 下载图像质量评估任务的测试集
test_dataset = load_dataset("q-future/q-eval-plus",
name="image_quality",
split="test")

# 访问数据集样本
for example in train_dataset:
print(example)

### 方法2：按任务下载全部划分
通过指定`split`参数，可下载特定任务的所有数据集划分：
python
from datasets import load_dataset

# 图像质量评估任务：训练集与测试集
img_quality_train = load_dataset("q-future/q-eval-plus", name="image_quality", split="train")
img_quality_test = load_dataset("q-future/q-eval-plus", name="image_quality", split="test")

# 图像-文本对齐评估任务：训练集与测试集
img_align_train = load_dataset("q-future/q-eval-plus", name="image_alignment", split="train")
img_align_test = load_dataset("q-future/q-eval-plus", name="image_alignment", split="test")

# 视频质量评估任务：训练集与测试集
vid_quality_train = load_dataset("q-future/q-eval-plus", name="video_quality", split="train")
vid_quality_test = load_dataset("q-future/q-eval-plus", name="video_quality", split="test")

# 视频-文本对齐评估任务：训练集与测试集
vid_align_train = load_dataset("q-future/q-eval-plus", name="video_alignment", split="train")
vid_align_test = load_dataset("q-future/q-eval-plus", name="video_alignment", split="test")

### 方法3：批量下载全部数据集
可一次性下载所有任务的数据集：
python
from datasets import load_dataset

# 定义所有任务与划分组合
tasks = [
("image_quality", "train"),
("image_quality", "test"),
("image_alignment", "train"),
("image_alignment", "test"),
("video_quality", "train"),
("video_quality", "test"),
("video_alignment", "train"),
("video_alignment", "test"),
]

datasets = {}
for task, split in tasks:
dataset = load_dataset("q-future/q-eval-plus", name=task, split=split)
datasets[f"{task}_{split}"] = dataset
print(f"已下载 {task} 任务的 {split} 集")

### 方法4：使用命令行工具
通过Hugging Face CLI下载数据集：
bash
# 若未安装Hugging Face Hub库，请先执行安装
pip install huggingface_hub

# 将所有数据集文件下载至本地目录
huggingface-cli download q-future/q-eval-plus --repo-type dataset --local-dir ./q-eval-plus

## 数据集格式
每个数据集文件均为包含评估样本的JSON数组，示例结构如下：
json
[
[
{
"model": "sd3.0-medium",
"prompt": "展示着一款蛋糕的面包店橱窗……",
"gt_score": 5.0,
"image_path": "Images/sd3.0-medium/image.png"
},
{
"model": "wanx-en",
"prompt": "展示着一款蛋糕的面包店橱窗……",
"gt_score": 3.3,
"image_path": "Images/wanx-en/image.png"
},
{
"choices": [
{
"type": "single",
"question": "核心评价依据是什么……",
"options": [...],
"answer": 0
}
]
}
]
]

### 字段说明
- **model**：用于生成内容的模型名称
- **prompt**：用于生成内容的文本提示词
- **gt_score**：基准质量得分（Ground Truth Score）
- **image_path** / **video_path**：生成内容的存储路径
- **choices**：用于评估的多项选择题
- **type**：题目类型（`single`代表单选题）
- **question**：评估问题文本
- **options**：候选答案列表
- **answer**：正确答案的索引

## 安装依赖
bash
# 安装所需依赖包
pip install datasets
pip install huggingface_hub

## 数据集统计
| 任务类型 | 训练集样本数 | 测试集样本数 |
|----------------------|--------------|--------------|
| 图像-文本对齐评估 | - | - |
| 图像质量评估 | - | - |
| 视频-文本对齐评估 | - | - |
| 视频质量评估 | - | - |

## 引用
若您在研究中使用本数据集，请引用如下文献：
bibtex
@dataset{qeval_plus_2024,
title={Q-Eval Plus},
author={Q-Future},
year={xx},
url={https://huggingface.co/datasets/q-future/q-eval-plus}
}

## 许可证

## 联系方式
如有疑问或问题，请访问[数据集仓库](https://huggingface.co/datasets/q-future/q-eval-plus)。

## 相关链接
- [Hugging Face 数据集文档](https://huggingface.co/docs/datasets/)
- [Q-Eval Plus GitHub 仓库](https://github.com/Q-Future/Q-Eval-plus)
- [模型评估基准平台](https://huggingface.co/q-future)