MagicBrush

# Dataset Card for MagicBrush

## Dataset Description
- **Homepage:** https://osu-nlp-group.github.io/MagicBrush
- **Repository:** https://github.com/OSU-NLP-Group/MagicBrush
- **Point of Contact:** [Kai Zhang](mailto:[e-mail])

### Dataset Summary

MagicBrush is the first large-scale, manually-annotated instruction-guided image editing dataset covering diverse scenarios single-turn, multi-turn, mask-provided, and mask-free editing. MagicBrush comprises 10K (source image, instruction, target image) triples, which is sufficient to train large-scale image editing models.

Please check our [website](https://osu-nlp-group.github.io/MagicBrush/) to explore more visual results.

#### Dataset Structure
"img_id" (str): same from COCO id but in string type, for easier test set loading

"turn_index" (int32): the edit turn in the image

"source_img" (str): input image, could be the original real image (turn_index=1) and edited images from last turn (turn_index >=2)

"mask_img" (str): free-form mask image (white region), can be used in mask-provided setting to limit the region to be edited.

"instruction" (str): edit instruction of how the input image should be changed.

"target_img" (str): the edited image corresponding to the input image and instruction.

If you need auxiliary data, please use [training set](https://buckeyemailosu-my.sharepoint.com/:u:/g/personal/zhang_13253_buckeyemail_osu_edu/EYEqf_yG36lAgiXw2GvRl0QBDBOeZHxvNgxO0Ec9WDMcNg) and [dev set](https://buckeyemailosu-my.sharepoint.com/:u:/g/personal/zhang_13253_buckeyemail_osu_edu/EXkXvvC95C1JsgMNWGL_RcEBElmsGxXwAAAdGamN8PNhrg)

### Splits

train: 8,807 edit turns (4,512 edit sessions).
dev: 528 edit turns (266 edit sessions).
test: (To prevent potential data leakage, please check our repo for information on obtaining the test set.)

### Licensing Information

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.

## Citation Information

If you find this dataset useful, please consider citing our paper:

```
@inproceedings{Zhang2023MagicBrush,
title={MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editing},
author={Kai Zhang and Lingbo Mo and Wenhu Chen and Huan Sun and Yu Su},
booktitle={Advances in Neural Information Processing Systems},
year={2023}
}
```

# MagicBrush 数据集卡片

## 数据集说明
- **项目主页：** https://osu-nlp-group.github.io/MagicBrush
- **代码仓库：** https://github.com/OSU-NLP-Group/MagicBrush
- **联系人：** [张凯（Kai Zhang）](mailto:[e-mail])

### 数据集概述
MagicBrush是首个大规模、经人工标注的指令引导型图像编辑数据集，涵盖单轮、多轮、带掩码及无掩码编辑等多元场景。该数据集包含10000组（源图像、编辑指令、目标图像）三元组，规模足以支撑大规模图像编辑模型的训练。

请访问我们的[项目主页](https://osu-nlp-group.github.io/MagicBrush/)以查看更多可视化结果。

#### 数据集结构
`"img_id"`（字符串类型）：与COCO（Common Objects in Context）数据集的图像ID保持一致，但采用字符串格式以简化测试集加载流程
`"turn_index"`（32位整数）：图像的编辑轮次索引
`"source_img"`（字符串类型）：输入图像，当`turn_index=1`时为原始真实图像，当`turn_index≥2`时为上一轮编辑后的图像
`"mask_img"`（字符串类型）：自由格式掩码图像（白色区域为待编辑范围），可在带掩码的编辑场景中用于限定编辑区域
`"instruction"`（字符串类型）：指导输入图像如何修改的编辑指令
`"target_img"`（字符串类型）：与输入图像及编辑指令对应的最终编辑后图像

若需辅助数据，请使用[训练集](https://buckeyemailosu-my.sharepoint.com/:u:/g/personal/zhang_13253_buckeyemail_osu_edu/EYEqf_yG36lAgiXw2GvRl0QBDBOeZHxvNgxO0Ec9WDMcNg)与[开发集](https://buckeyemailosu-my.sharepoint.com/:u:/g/personal/zhang_13253_buckeyemail_osu_edu/EXkXvvC95C1JsgMNWGL_RcEBElmsGxXwAAAdGamN8PNhrg)。

### 数据集划分
训练集：共8807个编辑轮次，涵盖4512个编辑会话。
开发集：共528个编辑轮次，涵盖266个编辑会话。
测试集：为防止潜在的数据泄露，请参阅我们的代码仓库以获取测试集的获取方式。

### 许可协议信息
知识共享许可协议
本作品采用知识共享署名4.0国际许可协议进行许可。

## 引用信息
若您认为本数据集对您的研究有所帮助，请引用我们的论文：

@inproceedings{Zhang2023MagicBrush,
title={MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editing},
author={Kai Zhang and Lingbo Mo and Wenhu Chen and Huan Sun and Yu Su},
booktitle={Advances in Neural Information Processing Systems},
year={2023}
}

搜集汇总

背景与挑战

背景概述

MagicBrush是一个大规模、手动标注的指令引导图像编辑数据集，包含10K个（源图像、指令、目标图像）三元组，覆盖多种编辑场景，适用于训练大规模图像编辑模型。数据集提供了详细的字段描述和划分信息，可用于多种图像编辑任务的研究和开发。