haizelabs/mj1-training-clean

---
license: cc-by-nc-4.0
task_categories:
- image-text-to-text
- visual-question-answering
language:
- en
tags:
- multimodal
- reward-model
- judge
- preference-data
- vision-language
pretty_name: MJ1 Training Data
size_categories:
- 10K<n<100K
---

# MJ1 Training Data

Training data for **MJ1 (MultiModal Judge 1)** - a multimodal reward model for evaluating vision-language model outputs.

## Dataset Summary

MJ1 Training Data is a curated, multi-source preference dataset designed for training a multimodal judge capable of evaluating responses across text and image modalities. Every datapoint contains at least one image and covers three distinct evaluation scenarios:

1. **Prompt image + text responses** (`reason`) - Given an image and a question, judge which text response is better.
2. **Image responses, no prompt image** (`t2i`) - Given a text prompt, judge which generated image is better.
3. **Prompt image + image responses** (`edit`) - Given a source image and an edit instruction, judge which edited image is better.

## Sources

| Source | Category | Scenario | Datapoints | Filtering |
|---|---|---|---|---|
| [Rapidata/human-coherence-preferences-images](https://huggingface.co/datasets/Rapidata/human-coherence-preferences-images) | `t2i` | 2 images (image responses) | 12,338 | Winner vote % >= 70% |
| [TIGER-Lab/EditReward-Data](https://huggingface.co/datasets/TIGER-Lab/EditReward-Data) | `edit` | 3 images (prompt + image responses) | 38,303 | Score differential >= 3.0 |
| [openbmb/RLAIF-V-Dataset](https://huggingface.co/datasets/openbmb/RLAIF-V-Dataset) | `reason` | 1 image (prompt image + text responses) | 20,000 | First 20k valid rows (responses >= 10 chars) |

**Total: ~70,641 datapoints**

## Schema

Each datapoint follows this structure:

| Field | Type | Description |
|---|---|---|
| `id` | string | Unique identifier (e.g., `rapidata-00001`, `editreward-00001`, `rlaifv-00001`) |
| `prompt` | string | Text prompt or question. Always present. |
| `prompt_image` | string or null | Path to prompt image. Null when no prompt image exists. |
| `response_a_text` | string or null | Text response A. Present when responses are text, null otherwise. |
| `response_a_image` | string or null | Path to image response A. Present when responses are images, null otherwise. |
| `response_b_text` | string or null | Text response B. Present when responses are text, null otherwise. |
| `response_b_image` | string or null | Path to image response B. Present when responses are images, null otherwise. |
| `ground_truth` | string | `"a"` or `"b"` - which response is better. |
| `source` | string | Origin dataset identifier. |
| `category` | string | Task category: `t2i`, `edit`, or `reason`. |

## Data Quality

- All images are **1024x1024 JPEG** format
- Ground truth A/B assignments are **balanced per-prompt** (within ±1) and approximately 50/50 overall
- Only high-confidence comparisons are included (filtered by vote margins or score differentials)
- Prompts are cleaned: no non-ASCII characters, no double spaces, no empty/garbage responses
- All image pairs are unique (no duplicate comparisons)

## Image Directory Structure

```
images/
prompts/ # Prompt images (edit + reason categories)
responses_a/ # Response A images (t2i + edit categories)
responses_b/ # Response B images (t2i + edit categories)
```

## Citations

```bibtex
@dataset{rapidata2024coherence,
title={Human-Coherence-Preferences-Images},
author={Rapidata},
year={2024},
publisher={Hugging Face},
url={https://huggingface.co/datasets/Rapidata/human-coherence-preferences-images}
}
```

```bibtex
@article{editreward2025,
title={EditReward: A Human-Aligned Reward Model for Instruction-Guided Image Editing},
author={TIGER-Lab},
journal={arXiv preprint arXiv:2509.26346},
year={2025}
}
```

```bibtex
@article{rlaifv2024,
title={RLAIF-V: Open-Source AI Feedback Leads to Super GPT-4V Trustworthiness},
author={OpenBMB},
journal={arXiv preprint arXiv:2405.17220},
year={2024}
}
```