infovqa_colqwen2_embeddings

# InfoVQA ColQwen2.5 Embeddings This dataset contains pre-computed embeddings for the InfoVQA dataset using the [ColQwen2.5](https://huggingface.co/vidore/colqwen2.5-v0.2) model. ## Dataset Structure The dataset consists of three configurations: ### Corpus Configuration Contains document images with their embeddings. ```python from datasets import load_dataset corpus = load_dataset("WenxingZhu/infovqa_colqwen2_embeddings", "corpus", split="test") ``` **Fields:** - `corpus-id` (int): Document identifier - `image` (Image): Original document image - `embeddings` (List[List[float]]): ColQwen2.5 image embeddings - `num_patches` (int): Number of image patches - `embedding_dim` (int): Embedding dimension (128) ### Queries Configuration Contains text queries with their embeddings. ```python from datasets import load_dataset queries = load_dataset("WenxingZhu/infovqa_colqwen2_embeddings", "queries", split="test") ``` **Fields:** - `query-id` (int): Query identifier - `query` (str): Query text - `embeddings` (List[List[float]]): ColQwen2.5 query embeddings - `num_tokens` (int): Number of tokens - `embedding_dim` (int): Embedding dimension (128) ### Qrels Configuration Contains query-document relevance judgments for evaluation. ```python from datasets import load_dataset qrels = load_dataset("WenxingZhu/infovqa_colqwen2_embeddings", "qrels", split="test") ``` **Fields:** - `query-id` (int): Query identifier - `corpus-id` (int): Document identifier - `score` (int): Relevance score (typically 0 or 1) ## Usage Example ```python from datasets import load_dataset import numpy as np # Load datasets corpus = load_dataset("WenxingZhu/infovqa_colqwen2_embeddings", "corpus", split="test") queries = load_dataset("WenxingZhu/infovqa_colqwen2_embeddings", "queries", split="test") qrels = load_dataset("WenxingZhu/infovqa_colqwen2_embeddings", "qrels", split="test") # Access embeddings corpus_embedding = np.array(corpus[0]["embeddings"]) query_embedding = np.array(queries[0]["embeddings"]) # Compute similarity (MaxSim strategy) def maxsim_similarity(query_emb, doc_emb): scores = np.dot(query_emb, doc_emb.T) max_scores = np.max(scores, axis=1) return np.mean(max_scores) similarity = maxsim_similarity(query_embedding, corpus_embedding) print(f"Similarity: {similarity}") # Check relevance for a query-document pair query_id = queries[0]["query-id"] corpus_id = corpus[0]["corpus-id"] relevant = qrels[(qrels["query-id"] == query_id) & (qrels["corpus-id"] == corpus_id)] print(f"Relevant: {len(relevant) > 0}") ``` ## Dataset Statistics - **Corpus size**: 500 documents - **Queries size**: 494 queries - **Qrels size**: 500 query-document pairs - **Source**: [InfoVQA Test Subsampled BEIR](https://huggingface.co/datasets/vidore/infovqa_test_subsampled_beir) - **Model**: [ColQwen2.5-v0.2](https://huggingface.co/vidore/colqwen2.5-v0.2) ## Citation ```bibtex @misc{faysse2024colpali, title={ColPali: Efficient Document Retrieval with Vision Language Models}, author={Manuel Faysse and Hugues Sibille and Tony Wu and Bilel Omrani and Gautier Viaud and Céline Hudelot and Pierre Colombo}, year={2024}, eprint={2407.01449}, archivePrefix={arXiv} } ```