Chrisyichuan/moca-visrag-ind-training

---
license: mit
task_categories:
- image-retrieval
- question-answering
language:
- en
pretty_name: MOCA VisRAG Independent Training
size_categories:
- 100K<n<1M
---

# Chrisyichuan/moca-visrag-ind-training

MOCA VisRAG independent-split contrastive training data with hard negatives.

## Contents

- `moca_visrag_ind_converted.jsonl` — query-image pairs with hard negatives
- `images/` — all referenced images

Each metadata row:

```json
{
"query": "...",
"chunk_path": "images/...",
"neg_chunk_paths": ["images/...", "images/..."],
"source_positive_rank": 0,
"source_positive_score": 0.0,
"source_dataset": "moca"
}
```

## Summary

- rows: 122752
- unique images: 122752
- avg negatives/row: 2.00

## Download

```python
from huggingface_hub import snapshot_download
snapshot_download(repo_id="Chrisyichuan/moca-visrag-ind-training", repo_type="dataset", local_dir="data/moca-visrag-ind-training")
```

## Image Storage

Images are stored as **123 tar shards** under `image_shards/` for fast download.

After cloning/downloading, extract images:

```bash
python extract_hf_image_shards.py --dataset-dir .
```

This creates `images/` with all referenced image files.