AliN96/midjourney-prompts-embeddings

---
license: mit
---

# Midjourney Prompt–Embedding Dataset

This dataset is derived from our COLM 2024 paper, *[Iteratively Prompting Multimodal LLMs to Reproduce Natural and AI-Generated Images](https://openreview.net/pdf?id=SwUsFTtM9h)*. The paper studies whether multimodal language models can infer prompts that generate images visually similar to target images produced by text-to-image systems or found in stock image collections, highlighting the relationship between real-world prompts and generated images as well as broader economic and security implications.

The dataset consists of a processed subset of approximately 4.5 million samples collected from publicly accessible Midjourney Discord channels. Each sample includes an anonymized prompt, the corresponding Midjourney model version, and a mapping to a precomputed image embedding. Embeddings are computed using CLIP (`laion/CLIP-ViT-g-14-laion2B-s12B-b42K`) and stored separately in shard files.

⚠️ Raw images are not redistributed due to platform terms of service, copyright considerations, and potential misuse risks.

## Accessing Image Embeddings

The dataset provides a mapping to precomputed image embeddings stored in shard files (`.npy`). These embeddings are not loaded automatically with `load_dataset` and should be accessed separately using the `EmbeddingShard` and `EmbeddingIndex` fields.

### Option 1: Download a single embedding shard

```python
from datasets import load_dataset
from huggingface_hub import hf_hub_download
import numpy as np

ds = load_dataset("AliN96/midjourney-prompts-embeddings", split="train")
sample = ds[0]

file_path = hf_hub_download(
repo_id="AliN96/midjourney-prompts-embeddings",
filename=sample["EmbeddingShard"], # e.g., embeddings_shard_00000_start_0_end_200000.npy
repo_type="dataset",
cache_dir="./hf_cache" # optional
)

embeddings = np.load(file_path)
vector = embeddings[sample["EmbeddingIndex"]]
```

### Option 2: Download all embedding shards

```python
from huggingface_hub import snapshot_download

snapshot_download(
repo_id="AliN96/midjourney-prompts-embeddings",
repo_type="dataset",
allow_patterns="*.npy",
local_dir="./embeddings"
)
```

You can change `local_dir` to control where the embedding files are stored locally.

👉 For full details on data collection, preprocessing, and usage, please refer to the GitHub repository:
[GitHub Repository](https://github.com/SPIN-UMass/MidjourneyDataset)

---

## Citation

If you use this dataset, please cite:

```bibtex
@inproceedings{
naseh2024iteratively,
title={Iteratively Prompting Multimodal {LLM}s to Reproduce Natural and {AI}-Generated Images},
author={Ali Naseh and Katherine Thai and Mohit Iyyer and Amir Houmansadr},
booktitle={First Conference on Language Modeling},
year={2024},
url={https://openreview.net/forum?id=SwUsFTtM9h}
}