a1557811266/Inter-Edit-Test

# Inter-Edit-Test

Official test benchmark release for the CVPR 2026 paper:

**Inter-Edit: First Benchmark for Interactive Instruction-Based Image Editing**

This repository hosts the public release of **Inter-Edit-Test**, a human-annotated benchmark for the Interactive Instruction-based Image Editing (`I^3E`) task.

Each sample contains:

- a source image,
- a coarse user-style interaction mask,
- a concise editing instruction,
- and a ground-truth edited image.

To simplify large-scale distribution on Hugging Face, image assets are packaged as tar archives while metadata remains directly accessible.

## Highlights

- **6,250** human-annotated test pairs
- **Bilingual instructions**: English and Chinese
- **User-style coarse masks**, rather than segmentation-perfect masks
- **High-resolution benchmark**
- **Challenging subsets** including artistic styles, low-resolution images, low-aesthetic images, and ambiguous multi-instance edits
- **Sanitized public release**: all filenames are re-indexed to remove source-specific naming information

## Release Statistics

### Language distribution

- English: **3,413**
- Chinese: **2,837**

### Edit type distribution

- Remove: **1,641**
- Add: **1,603**
- Local: **1,533**
- Text Editing: **1,084**
- Texture: **389**

## Repository Structure

```text
Inter-Edit-Test-HF/
├── source_images.tar
├── target_images.tar
├── masks.tar
├── metadata.json
├── metadata.jsonl
└── README.md
```

Archive contents use index-based sanitized names:

- `source_images/00000_source.xxx`
- `target_images/00000_gt.xxx`
- `masks/00000_mask.xxx`

## Metadata Format

Each record in `metadata.json` / `metadata.jsonl` contains:

- `index`: zero-based dataset index
- `source_path`: relative path inside the extracted archive
- `gt_path`: relative path inside the extracted archive
- `mask_path`: relative path inside the extracted archive
- `edit_type`: edit category
- `language`: instruction language
- `instruction`: concise editing instruction

## Quick Start

Extract the archives after download:

```bash
tar -xf source_images.tar
tar -xf target_images.tar
tar -xf masks.tar
```

Then load metadata as usual:

```python
import json
from pathlib import Path

root = Path(".")
with open(root / "metadata.json", "r", encoding="utf-8") as f:
meta = json.load(f)

sample = meta[0]
source = root / sample["source_path"]
target = root / sample["gt_path"]
mask = root / sample["mask_path"]
instruction = sample["instruction"]
```

## Benchmark Intent

Inter-Edit-Test evaluates whether a model can:

1. correctly understand a concise edit instruction,
2. localize the intended region from an imprecise user mask,
3. perform the desired edit inside the target area,
4. preserve and harmonize the surrounding image naturally.

## Project Page

- GitHub: https://github.com/Delong-liu-bupt/Inter-Edit

## Citation

If you use Inter-Edit-Test in your research, please cite:

```bibtex
@inproceedings{liu2026interedit,
title = {Inter-Edit: First Benchmark for Interactive Instruction-Based Image Editing},
author = {Liu, Delong and Hou, Haotian and Hou, Zhaohui and Huang, Zhiyuan and Han, Shihao and Zhan, Mingjie and Zhao, Zhicheng and Su, Fei},
booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
year = {2026}
}
```