VisRAG-Ret-Test-ArxivQA

## Dataset Description
This is a VQA dataset based on figures extracted from arXiv publications taken from ArXiVQA dataset from [Multimodal ArXiV](https://arxiv.org/abs/2403.00231).

### Load the dataset
```python
from datasets import load_dataset
import csv

def load_beir_qrels(qrels_file):
qrels = {}
with open(qrels_file) as f:
tsvreader = csv.DictReader(f, delimiter="\t")
for row in tsvreader:
qid = row["query-id"]
pid = row["corpus-id"]
rel = int(row["score"])
if qid in qrels:
qrels[qid][pid] = rel
else:
qrels[qid] = {pid: rel}
return qrels

corpus_ds = load_dataset("openbmb/VisRAG-Ret-Test-ArxivQA", name="corpus", split="train")
queries_ds = load_dataset("openbmb/VisRAG-Ret-Test-ArxivQA", name="queries", split="train")

qrels_path = "xxxx" # path to qrels file which can be found under qrels folder in the repo.
qrels = load_beir_qrels(qrels_path)
```

## 数据集说明
本数据集为基于arXiv预印本论文中提取的图表构建的视觉问答（Visual Question Answering, VQA）数据集，其数据源自[Multimodal ArXiV](https://arxiv.org/abs/2403.00231)一文提出的ArXiVQA数据集。

### 数据集加载代码
python
from datasets import load_dataset
import csv

def load_beir_qrels(qrels_file):
qrels = {}
with open(qrels_file) as f:
tsvreader = csv.DictReader(f, delimiter=" ")
for row in tsvreader:
qid = row["query-id"]
pid = row["corpus-id"]
rel = int(row["score"])
if qid in qrels:
qrels[qid][pid] = rel
else:
qrels[qid] = {pid: rel}
return qrels

corpus_ds = load_dataset("openbmb/VisRAG-Ret-Test-ArxivQA", name="corpus", split="train")
queries_ds = load_dataset("openbmb/VisRAG-Ret-Test-ArxivQA", name="queries", split="train")

qrels_path = "xxxx" # 查询-文档相关度标注文件路径，可在仓库的qrels文件夹中获取
qrels = load_beir_qrels(qrels_path)

搜集汇总

背景与挑战

背景概述

该数据集是一个基于ArXiv论文图表构建的视觉问答（VQA）测试集，源自ArXivQA数据集，主要用于评估检索增强生成（RAG）系统的性能。