vaqa/rag-qa

---
dataset_info:
features:
- name: query
dtype: string
- name: answer
dtype: string
- name: category
dtype: string
- name: source
dtype: string
- name: context_id
dtype: int64
splits:
- name: train
num_bytes: 14162422
num_examples: 39435
- name: boolean
num_bytes: 11260
num_examples: 100
- name: complex_qa
num_bytes: 29871
num_examples: 100
- name: core
num_bytes: 26380
num_examples: 100
- name: math_basic
num_bytes: 24019
num_examples: 100
- name: multipart
num_bytes: 96209
num_examples: 100
- name: not_found_classification
num_bytes: 19738
num_examples: 100
- name: summary
num_bytes: 41658
num_examples: 100
download_size: 5742127
dataset_size: 14411557
configs:
- config_name: default
data_files:
- split: train
path: data/train-*
- split: boolean
path: data/boolean-*
- split: complex_qa
path: data/complex_qa-*
- split: core
path: data/core-*
- split: math_basic
path: data/math_basic-*
- split: multipart
path: data/multipart-*
- split: not_found_classification
path: data/not_found_classification-*
- split: summary
path: data/summary-*
---

### Dataset Columns

`context_id` is a reference to the context in https://huggingface.co/datasets/barnwell/rag-kb

`source` is a reference to the one of these datasets:
- https://huggingface.co/datasets/llmware/rag_instruct_benchmark_tester
- https://huggingface.co/datasets/virattt/financial-qa-10K
- https://huggingface.co/datasets/dariolopez/justicio-rag-embedding-qa-tmp-2
- https://huggingface.co/datasets/glaiveai/RAG-v1
- https://huggingface.co/datasets/neural-bridge/rag-dataset-12000
- https://huggingface.co/datasets/neural-bridge/rag-hallucination-dataset-1000
- https://huggingface.co/datasets/lighteval/natural_questions_clean

`category` classifies the query into one one of these categories:
- **Core Q&A Evaluation** - fact-based 'core' questions- used to assign a score between 0-100 based on correct responses.
- **Not Found Classification** - in each sample, the context passage does not contain a direct answer to the question, and the objective is to evaluate whether the model correctly identifies as "Not Found" or attempts to answer using information in the context.
- **Boolean - Yes/No** - each sample is a Yes/No question.
- **Basic Math** - these are "every day" math questions - basic increments, decrements, percentages, multiplications, sorting, and ranking with amounts and times.
- **Complex Q&A** - tests several distinct 'complex q&a' skills - multiple-choice, financial table reading, multi-part extractions, causal, and logical selections.
- **Summary** - tests long-form and short-form summarization.
- **Multi-Part** - Multiple questions that can be answered by a single context.