vidore_v3_computer_science

<center><h1>ViDoRe V3 : Computer Science</h1></center>

This dataset, `Computer Science`, is a corpus of `textbooks` from the openstacks website, intended for long-document understanding tasks. It is one of the 10 corpora comprising the **ViDoRe v3 Benchmark**.

## About ViDoRe v3

[](https://arxiv.org/abs/2601.08620)

ViDoRe V3 is our latest benchmark for RAG evaluation on visually-rich documents from real-world applications. It features 10 datasets with, in total, 26,000 pages and 3099 queries, translated into 6 languages. Each query comes with human-verified relevant pages, bounding box annotations for key elements, and a comprehensive combined answer from human annotations.

## Links

* **Homepage:** [https://huggingface.co/vidore](https://huggingface.co/vidore)
* **Collection:** [https://hf.co/collections/vidore/vidore-benchmark-v3](https://hf.co/collections/vidore/vidore-benchmark-v3)
* **Preprint:** [https://arxiv.org/abs/2601.08620](https://arxiv.org/abs/2601.08620)
* **Blogpost:** [https://huggingface.co/blog/QuentinJG/introducing-vidore-v3](https://huggingface.co/blog/QuentinJG/introducing-vidore-v3)
* **Leaderboard:** [https://huggingface.co/spaces/vidore/vidore-leaderboard](https://huggingface.co/spaces/vidore/vidore-leaderboard)

### Dataset Summary

This specific dataset (`Computer Science`) contains documents related to textbook classes on computer science.

- Description: Consists of two books from the openstacks website

- Language: en

- Domain: Computer Science

- Document Types: TextBooks

>[!NOTE]
> Commit used for end-to-end evaluation: `7b91f10e18b72a763dd17a0c05d66bf985b98f1d`

### Dataset Statistics

- Total Documents : 2
- Total Pages : 1360
- Total Queries : 1290
- Queries without counting translations : 215
- Average number of pages per query : 4.6

### Languages

The documents in this dataset are in `english`.

### Queries type


### Queries format


### Content type


## Dataset Structure

### 1. Corpus

Contains the full collection of documents to be searched.

Data instance of a single item from the corpus subset:

```json
{
"corpus_id": <int>,
"image": <PIL.Image>,
"doc_id": <str>,
"markdown": <str>,
"page_number_in_doc": <int>
}
```

- **corpus_id** <int> : A unique numerical identifier for the corresponding corpus document.
- **image** <PIL.Image> : The page
- **doc_id** <str> : name of the document from where the image was extracted
- **markdown** <str> : Extracted text from the Image using an OCR pipeline
- **page_number_in_doc** <int> : Original page number inside the document

### 2. Queries
Contains set of questions or search queries.

Data Instance of a single item from the queries subset:

```json
{
"query_id": <int>,
"query": <str>,
"language": <str>,
"query_types": <List[str]>,
"query_format": <str>,
"content_type": <str>,
"raw_answers": <List[str]>,
"query_generator": <str>,
"query_generation_pipeline": <str>,
"source_type": <str>,
"query_type_for_generation": <str>,
"answer": <str>
}
```

- **query_id** <int> : A unique numerical identifier for the query.
- **query** <str> : The actual text of the search question or statement used for retrieval.
- **language** <str> : The language of the query text.
- **query_types** <List[str]> : A list of categories or labels describing the query's intent.
- **query_format** <str> : The syntactic format of the query ("intruction", "keyword" or "question").
- **content_type** <str> : The type of visual content present images relevant for the query.
- **raw_answers** <List[str]> : A list of reference answers written by human annotators.
- **query_generator** <str> : The source or method used to create the query ("human" or "sdg").
- **query_generation_pipeline** <str> : Type of SDG pipeline used to create the query (if it was not written by humans)
- **source_type** <str> : "summary" or "image", metadata about the type of information used by the annotation pipeline to create the query
- **query_type_for_generation** <str> : The specific type requested when the query was generated
- **answer** <str> : The answer extracted from the source documents, merged from human annotations using an LLM.

### 3. Qrels
Maps queries to their corresponding relevant documents.

Data Instance of a single item for the qrels subset:
```json
{
"query_id": <int>,
"corpus_id": <int>,
"score": <int>,
"content_type": <str>,
"bounding_boxes": <List[Tuple[int]]>
}
```

- **query_id** <int> : A unique numerical identifier for the query.
- **corpus_id** <int> : A unique numerical identifier for the corresponding corpus document.
- **score** <int> : Relevance score for the pair `<query, corpus>`. Can be either 1 (Critically Relevant) or 2 (Fully Relevant):
- Fully Relevant (2) - The page contains the complete answer.
- Critically Relevant (1) - The page contains facts or information that are required to answer the query, though additional information is required.
- **content_type** <str> : The type of visual content present images relevant for the query.
- **bounding_boxes** <List[Tuple[int]]> : Bounding boxes annotated by humans that indicate which part of the image is relevant to the query.

### 4. Original PDFs
All the original pdfs used to build the corpus are distributed in the "pdfs" folder of this directory.

## License information

All annotations, query-document relevance judgments (qrels), and related metadata generated for this corpus are distributed under the Creative Commons Attribution 4.0 International License (CC BY 4.0).

The licensing status of the original source documents (the corpus) and any parsed text (`markdown` column in the corpus) are inherited from their respective publishers. The specific license governing each original document is provided in the `documents_metadata["license"]` field of that document's entry.

## Data Privacy and Removal Requests

While this dataset is released under open licenses, we respect the privacy of individuals and the ownership of source content. If you are a data subject, author, or publisher and are uncomfortable with the inclusion of your data or documents in this release, please contact us at [e-mail] and [e-mail]. We will promptly review your request.