racineai/VDR_MEGA_MultiDomain_DocRetrieval

---
task_categories:
- visual-document-retrieval
- text-retrieval
language:
- en
- fr
- de
- es
- it
multilinguality:
- multilingual
size_categories:
- 1M<n<10M
modalities:
- image
- text
formats:
- parquet
library_name:
- datasets
- dask
- croissant
- polars
tags:
- document-retrieval
- RAG
- DSE
- retrieval
configs:
- config_name: default
data_files: '**/*.parquet'
license: apache-2.0
---
# Visual Document Retrieval Dataset

## Overview
This dataset is designed for training visual document retrieval models. It combines multiple datasets from the VDR series, Colpali, and LlamaIndex to create the most comprehensive training resource for visual document retrieval tasks.

## Dataset Structure
The dataset contains structured fields including unique identifiers with string lengths ranging from 45 to 50 characters, search query text with variable lengths between 5 and 336 characters, and language classifications across 5 distinct values. Each entry includes a number of negative examples ranging from 0 to 16 integers, accompanied by a primary document image with widths spanning 366 to 5310 pixels. Additional negative example images are provided through fields negative_image_0 to negative_image_15, featuring widths between 622 and 827 pixels.

## Language Distribution

The dataset encompasses content across five languages with approximately 1,090,000 total examples:

| Language | Examples | Percentage |
|----------|----------|------------|
| English (en) | ~700,770 | 64.3% |
| French (fr) | ~224,540 | 20.6% |
| German (de) | ~56,680 | 5.2% |
| Spanish (es) | ~56,680 | 5.2% |
| Italian (it) | ~52,320 | 4.8% |
| **Total** | **~1,090,000** | **100%** |

## Purpose
This dataset serves as a comprehensive training resource for visual document retrieval models by providing both positive and negative examples to enhance model discrimination capabilities. The dataset optimizes training efficiency by including examples with and without negative samples, allowing models to learn from diverse training scenarios. The multilingual composition ensures robust performance across different languages and diverse document types. The extensive negative sampling mechanism supports contrastive learning approaches essential for effective visual document retrieval model training.

## Data Sources
This dataset represents a strategic fusion of established datasets from multiple sources:

### VDR Series
- [racineai/VDR_Military](https://huggingface.co/datasets/racineai/VDR_Military) - Military domain documents (187k examples)
- [racineai/VDR_Energy](https://huggingface.co/datasets/racineai/VDR_Energy) - Energy sector documents (160k examples)
- [racineai/VDR_Geotechnie](https://huggingface.co/datasets/racineai/VDR_Geotechnie) - Geotechnical engineering documents (68.3k examples)
- [racineai/VDR_Hydrogen](https://huggingface.co/datasets/racineai/VDR_Hydrogen) - Hydrogen technology documents

### Visual Document Retrieval
- [vidore/colpali_train_set](https://huggingface.co/datasets/vidore/colpali_train_set) - Core training examples for visual document understanding
- [openbmb/VisRAG-Ret-Train-Synthetic-data](https://huggingface.co/datasets/openbmb/VisRAG-Ret-Train-Synthetic-data) - Synthetic visual retrieval training data
- [llamaindex/vdr-multilingual-train](https://huggingface.co/datasets/llamaindex/vdr-multilingual-train) - Multilingual training dataset for visual document retrieval
-
This consolidation creates the largest and most complete dataset currently available for visual document retrieval model training, combining the strengths and coverage of each contributing source to maximize training effectiveness.

## Citation

If you use this dataset in your research, please cite:

**Authors:** Léo Appourchaux, Noé Brandolini, Paul Lemaistre, André-Louis Rochet

```bibtex
@misc{appourchaux_brandolini_lemaistre_rochet2025,
title = {Visual Document Retrieval Dataset: A Comprehensive Multilingual Training Resource},
author = {Léo Appourchaux and Noé Brandolini and Paul Lemaistre and André-Louis Rochet},
year = {2025},
eprint = {XXXX.XXXXX},
archivePrefix = {arXiv},
primaryClass = {cs.IR},
url = {https://arxiv.org/abs/XXXX.XXXXX}
}
```