DataDepictQA

# DataDepictQA

Datasets of the papers in [DepictQA project](https://depictqa.github.io/):

- DepictQA-Wild (DepictQA-v2): [paper](https://arxiv.org/abs/2405.18842) / [project page](https://depictqa.github.io/depictqa-wild/) / [code](https://github.com/XPixelGroup/DepictQA).

Zhiyuan You, Jinjin Gu, Zheyuan Li, Xin Cai, Kaiwen Zhu, Chao Dong, Tianfan Xue, "Descriptive Image Quality Assessment in the Wild," arXiv preprint arXiv:2405.18842, 2024.

- DepictQA-v1: [paper](https://arxiv.org/abs/2312.08962) / [project page](https://depictqa.github.io/depictqa-v1/) / [code](https://github.com/XPixelGroup/DepictQA).

Zhiyuan You, Zheyuan Li, Jinjin Gu, Zhenfei Yin, Tianfan Xue, Chao Dong, "Depicting beyond scores: Advancing image quality assessment through multi-modal language models," ECCV, 2024.

## Dataset Overview

- Training DepictQA-v2 requires:
- KADIS700K
- BAPPS
- PIPAL
- KADID10K
- DetailDescriptionLAMM

- Training DepictQA-v1 requires:
- BAPPS
- PIPAL
- KADID10K
- DetailDescriptionLAMM

## Dataset Construction

**Source codes** for dataset construction are provided in [here](https://github.com/XPixelGroup/DepictQA/tree/main/build_datasets).

Our datasets are constructed based on existing datasets. Therefore, some source images should be downloaded and re-arranged to construct the datasets. Bellow we provide a detailed instruction.

### KADIS700K

After downloading, extract the images from _*.tar.gz_ files. The directory of `DataDepictQA/KADIS700K` should be as follows.
The meanings of directory names can be found in **Abbreviations** section of our [source codes](https://github.com/XPixelGroup/DepictQA/tree/main/build_datasets) for dataset construction.

```
|-- DataDepictQA
|-- KADIS700K
|-- A_md_brief
|-- A_md_detail
|-- A_sd_brief
|-- A_sd_detail
|-- AB_md_detail
|-- AB_sd_detail
|-- metas_combine
|-- ref_imgs_s224
|-- refA_md_brief
|-- refA_md_detail
|-- refA_sd_brief
|-- refA_sd_detail
|-- refAB_md_detail
|-- refAB_sd_detail
```

### BAPPS

1. Download the BAPPS dataset (**2AFC Train set** and **2AFC Val set**) from [here](https://github.com/richzhang/PerceptualSimilarity/blob/master/scripts/download_dataset.sh).
2. Place the downloaded images in `DataDepictQA/BAPPS` as follows.
```
|-- DataDepictQA
|-- BAPPS
|-- images
|-- mbapps_test_refA_s64
|-- mbapps_test_refAB_s64
|-- twoafc_train (downloaded)
|-- twoafc_val (downloaded)
|-- resize_bapps.py
|-- metas
```
3. The downloaded images are 256 x 256 patches, which are resized from the original 64 x 64 patches.
Resizing does not influence comparison results (_i.e._, Image A or Image B is better), but influences the detailed reasoning tasks since additional pixelation distortion is introduced.
Therefore, we resize these images back to their original 64 x 64 resolution.
```
cd DataDepictQA/BAPPS/images
python resize_bapps.py
```
4. The constructed BAPPS directory should be as follows.
```
|-- DataDepictQA
|-- BAPPS
|-- images
|-- mbapps_test_refA_s64
|-- mbapps_test_refAB_s64
|-- twoafc_train (downloaded)
|-- twoafc_train_s64 (created by resize_bapps.py)
|-- twoafc_val (downloaded)
|-- twoafc_val_s64 (created by resize_bapps.py)
|-- resize_bapps.py
|-- metas
```

### PIPAL

1. Download the PIPAL dataset (**train set**) from [here](https://github.com/HaomingCai/PIPAL-dataset).
2. Place the downloaded images in `DataDepictQA/PIPAL` as follows.
```
|-- DataDepictQA
|-- PIPAL
|-- images
|-- Distortion_1 (downloaded)
|-- Distortion_2 (downloaded)
|-- Distortion_3 (downloaded)
|-- Distortion_4 (downloaded)
|-- Train_Ref (downloaded)
|-- metas
```

### KADID10K

1. Download the KADID10K dataset from [here](https://database.mmsp-kn.de/kadid-10k-database.html).
2. Place the downloaded images in `DataDepictQA/KADID10K` as follows.
```
|-- DataDepictQA
|-- KADID10K
|-- images/*.png (downloaded)
|-- metas
|-- metas_srcc_plcc_voting
```

### CSIQ

1. Download the CSIQ dataset from [here](https://s2.smu.edu/~eclarson/csiq.html).
2. Place the downloaded images in `DataDepictQA/CSIQ` as follows.
```
|-- DataDepictQA
|-- CSIQ
|-- images
|-- src_imgs (downloaded)
|-- dst_imgs (downloaded)
|-- metas
|-- metas_srcc_plcc_voting
```

### TID2013

1. Download the TID2013 dataset from [here](https://www.ponomarenko.info/tid2013.htm).
2. Place the downloaded images in `DataDepictQA/TID2013` as follows.
```
|-- DataDepictQA
|-- TID2013
|-- images
|-- reference_images (downloaded)
|-- distorted_images (downloaded)
|-- metas
|-- metas_srcc_plcc_voting
```

### DetailDescriptionLAMM

1. Download the LAMM Detailed Description dataset (**coco_images**) from [here](https://opendatalab.com/LAMM/LAMM/tree/main/raw/2D_Instruct).
2. Place the downloaded images in `DataDepictQA/DetailDescriptionLAMM` as follows.
```
|-- DataDepictQA
|-- DetailDescriptionLAMM
|-- coco_images/*.jpg (downloaded)
|-- metas
```

# DataDepictQA
本数据集为[DepictQA项目](https://depictqa.github.io/)相关论文的配套数据集集合：
- DepictQA-Wild（DepictQA-v2）：[论文](https://arxiv.org/abs/2405.18842) / [项目页面](https://depictqa.github.io/depictqa-wild/) / [代码](https://github.com/XPixelGroup/DepictQA)。
游志远、谷津津、李泽远、蔡昕、朱凯文、董超、薛天凡，《野外场景下的描述性图像质量评估》，arXiv预印本 arXiv:2405.18842，2024年。
- DepictQA-v1：[论文](https://arxiv.org/abs/2312.08962) / [项目页面](https://depictqa.github.io/depictqa-v1/) / [代码](https://github.com/XPixelGroup/DepictQA)。
游志远、李泽远、谷津津、尹振飞、薛天凡、董超，《超越分数：通过多模态大语言模型（Multi-modal Language Model）推进图像质量评估》，ECCV 2024。

## 数据集概览
- 训练DepictQA-v2需以下列数据集为基础：
- KADIS700K
- BAPPS
- PIPAL
- KADID10K
- DetailDescriptionLAMM
- 训练DepictQA-v1需以下列数据集为基础：
- BAPPS
- PIPAL
- KADID10K
- DetailDescriptionLAMM

## 数据集构建
**源代码** 数据集构建的配套源代码已在[此处](https://github.com/XPixelGroup/DepictQA/tree/main/build_datasets)公开。本数据集基于现有数据集构建，因此需下载部分源图像并进行重新整理，方可完成数据集构建。下文将提供详细操作说明。

### KADIS700K
下载完成后，从*.tar.gz文件中解压图像。`DataDepictQA/KADIS700K`的目录结构应如下所示。目录名称的含义可参见数据集构建源代码的[缩写说明部分](https://github.com/XPixelGroup/DepictQA/tree/main/build_datasets)。

|-- DataDepictQA
|-- KADIS700K
|-- A_md_brief
|-- A_md_detail
|-- A_sd_brief
|-- A_sd_detail
|-- AB_md_detail
|-- AB_sd_detail
|-- metas_combine
|-- ref_imgs_s224
|-- refA_md_brief
|-- refA_md_detail
|-- refA_sd_brief
|-- refA_sd_detail
|-- refAB_md_detail
|-- refAB_sd_detail

### BAPPS
1. 从[此处](https://github.com/richzhang/PerceptualSimilarity/blob/master/scripts/download_dataset.sh)下载BAPPS数据集（**2AFC（二择一强制选择，Two-Alternative Forced Choice）训练集**与**2AFC验证集**）。
2. 将下载的图像按如下结构放置在`DataDepictQA/BAPPS`目录中：

|-- DataDepictQA
|-- BAPPS
|-- images
|-- mbapps_test_refA_s64
|-- mbapps_test_refAB_s64
|-- twoafc_train (downloaded)
|-- twoafc_val (downloaded)
|-- resize_bapps.py
|-- metas

3. 下载的图像为256×256像素块，由原始64×64像素块缩放而来。缩放操作不会影响对比结果（即图像A与图像B的优劣判断），但会对细节推理任务产生影响，因为缩放会引入额外的像素化失真。因此，我们将这些图像重新缩放至原始的64×64分辨率。

cd DataDepictQA/BAPPS/images
python resize_bapps.py

4. 构建完成后的BAPPS目录结构应如下所示：

|-- DataDepictQA
|-- BAPPS
|-- images
|-- mbapps_test_refA_s64
|-- mbapps_test_refAB_s64
|-- twoafc_train (downloaded)
|-- twoafc_train_s64 (created by resize_bapps.py)
|-- twoafc_val (downloaded)
|-- twoafc_val_s64 (created by resize_bapps.py)
|-- resize_bapps.py
|-- metas

### PIPAL
1. 从[此处](https://github.com/HaomingCai/PIPAL-dataset)下载PIPAL数据集（**训练集**）。
2. 将下载的图像按如下结构放置在`DataDepictQA/PIPAL`目录中：

|-- DataDepictQA
|-- PIPAL
|-- images
|-- Distortion_1 (downloaded)
|-- Distortion_2 (downloaded)
|-- Distortion_3 (downloaded)
|-- Distortion_4 (downloaded)
|-- Train_Ref (downloaded)
|-- metas

### KADID10K
1. 从[此处](https://database.mmsp-kn.de/kadid-10k-database.html)下载KADID10K数据集。
2. 将下载的图像按如下结构放置在`DataDepictQA/KADID10K`目录中：

|-- DataDepictQA
|-- KADID10K
|-- images/*.png (downloaded)
|-- metas
|-- metas_srcc_plcc_voting

### CSIQ
1. 从[此处](https://s2.smu.edu/~eclarson/csiq.html)下载CSIQ数据集。
2. 将下载的图像按如下结构放置在`DataDepictQA/CSIQ`目录中：

|-- DataDepictQA
|-- CSIQ
|-- images
|-- src_imgs (downloaded)
|-- dst_imgs (downloaded)
|-- metas
|-- metas_srcc_plcc_voting

### TID2013
1. 从[此处](https://www.ponomarenko.info/tid2013.htm)下载TID2013数据集。
2. 将下载的图像按如下结构放置在`DataDepictQA/TID2013`目录中：

|-- DataDepictQA
|-- TID2013
|-- images
|-- reference_images (downloaded)
|-- distorted_images (downloaded)
|-- metas
|-- metas_srcc_plcc_voting

### DetailDescriptionLAMM
1. 从[此处](https://opendatalab.com/LAMM/LAMM/tree/main/raw/2D_Instruct)下载LAMM详细描述数据集（**coco_images**）。
2. 将下载的图像按如下结构放置在`DataDepictQA/DetailDescriptionLAMM`目录中：

|-- DataDepictQA
|-- DetailDescriptionLAMM
|-- coco_images/*.jpg (downloaded)
|-- metas