Chrisyichuan/moca-visrag-syn-training

--- license: mit task_categories: - image-retrieval - question-answering language: - en pretty_name: MOCA VisRAG Synthetic Training size_categories: - 100K<n<1M --- # Chrisyichuan/moca-visrag-syn-training MOCA VisRAG synthetic-split contrastive training data with hard negatives. ## Contents - `moca_visrag_syn_converted.jsonl` — query-image pairs with hard negatives - `images/` — all referenced images Each metadata row: ```json { "query": "...", "chunk_path": "images/...", "neg_chunk_paths": ["images/...", "images/..."], "source_positive_rank": 0, "source_positive_score": 0.0, "source_dataset": "moca" } ``` ## Summary - rows: 239206 - unique images: 239298 - avg negatives/row: 2.00 ## Download ```python from huggingface_hub import snapshot_download snapshot_download(repo_id="Chrisyichuan/moca-visrag-syn-training", repo_type="dataset", local_dir="data/moca-visrag-syn-training") ``` ## Image Storage Images are stored as **240 tar shards** under `image_shards/` for fast download. After cloning/downloading, extract images: ```bash python extract_hf_image_shards.py --dataset-dir . ``` This creates `images/` with all referenced image files.