orgrctera/uda_feta_qa_qa

---
license: cc-by-sa-4.0
language:
- en
pretty_name: UDA FeTa-QA (Question Answering)
size_categories:
- 1K<n<10K
tags:
- question-answering
- table-qa
- free-form
- wikipedia
- rag
- unstructured-documents
- uda-benchmark
configs:
- config_name: default
data_files:
- split: default
path: data/default-*
dataset_info:
features:
- name: input
dtype: string
- name: metadata
dtype: string
- name: answers
dtype: string
- name: doc_url
dtype: string
- name: evidence
dtype: string
splits:
- name: default
num_bytes: 2184719
num_examples: 1023
download_size: 889482
dataset_size: 2184719
---

# UDA FeTa-QA (`orgrctera/uda_feta_qa_qa`)

## Overview

This dataset is the **FeTa-QA** slice of the **UDA (Unstructured Document Analysis)** benchmark: **1,023** question–answer instances grounded in **Wikipedia** tables and accompanying context, packaged for **question answering** evaluation in RAG and document-analysis pipelines.

**UDA** ([Hui et al., 2024](https://arxiv.org/abs/2406.15187)) is a benchmark suite for Retrieval-Augmented Generation (RAG) over real-world documents in **PDF** and **HTML**, where evidence mixes narrative text and **tables**. In UDA, the **FetaTab** subset aligns with **FeTa-QA** ([Nan et al., TACL 2022](https://aclanthology.org/2022.tacl-1.3/)): **878** documents and **1,023** Q&A pairs labeled as **free-form** answers in the UDA paper’s statistics ([UDA-QA card](https://huggingface.co/datasets/qinchuanhui/UDA-QA)).

**FeTa-QA (FeTaQA)** introduces **free-form table question answering**: given a **table** (and its document context), the model must produce a **natural-language answer** that integrates multiple facts—often requiring retrieval over cells, relational reasoning, and **generative** explanation rather than a single short span.

This Hub release follows the same spirit as other UDA-derived exports: each row pairs a **natural-language question** with **structured supervision** (`expected_output`) that includes the reference answer, source **document URL**, and **evidence** (table snapshot and highlighted cell identifiers) for analysis or training.

## Task

- **Task type:** **Question answering (QA)** for **FeTa-QA**—**free-form** answers over **tabular** evidence (with Wikipedia-sourced documents in UDA’s original PDF/HTML setting).
- **Input:** A natural-language question (`input`) about entities, events, or relationships described in the table and page.
- **Supervision / reference:** `expected_output` is a JSON string with:
- **`answers`:** the gold free-form text answer;
- **`doc_url`:** Wikipedia URL for the source page;
- **`evidence`:** `table_array` (rows of the table as strings) and `highlighted_cell_ids` (row/column indices of supporting cells).

`metadata` records UDA identifiers (`sub_benchmark`: `feta_qa`, `benchmark_name`: `uda_feta_qa`).

Evaluation may combine **answer quality** (e.g. ROUGE/BERTScore against the reference free-form string, or human judgment) with **faithfulness** to retrieved table cells, following FeTa-QA and UDA protocols.

## Background

### FeTa-QA (FeTaQA)

Existing table QA datasets often use **short, extractive** answers, which under-represent questions that need **reasoning and integration** across multiple cells. **FeTa-QA** provides Wikipedia-based **{table, question, free-form answer, supporting cells}** pairs, requiring systems to **retrieve**, **infer**, and **compose** explanations involving entities and relations—not merely copy a single span ([Nan et al., 2021/2022](https://arxiv.org/abs/2104.00369)).

### UDA benchmark and the FetaTab slice

UDA revisits RAG and LLM-based document analysis with **2,965** real-world documents and **29,590** expert-annotated Q&A pairs, keeping sources in **original** formats to stress **parsing**, **chunking**, and **retrieval** alongside generation. The **FetaTab** row in UDA’s suite corresponds to FeTa-QA–aligned labeling at Wikipedia scale, with **free-form** Q&A types ([UDA paper](https://arxiv.org/abs/2406.15187), [UDA-Benchmark](https://github.com/qinchuanhui/UDA-Benchmark)).

## Data fields

| Column | Type | Description |
|--------|------|-------------|
| `input` | `string` | Question text posed over the table/page. |
| `expected_output` | `string` | JSON with `answers`, `doc_url`, and `evidence` (`table_array`, `highlighted_cell_ids`). |
| `metadata` | struct | `benchmark_name` (`uda_feta_qa`), `benchmark_type` (`uda`), `split`, `sub_benchmark` (`feta_qa`), and `value` (JSON string with identifiers such as `label_key`, `label_file`, `doc_url`, `q_uid`). |

**Splits:** Single split `default` with **1,023** examples.

## Examples

The following rows are illustrative of the dataset (JSON in `expected_output` is formatted for readability; large `table_array` values may be truncated in documentation).

**Example 1 — free-form answer over a stage credits table**

- **`input`:** `When and in what play did Platt appear at the Music Box Theatre?`
- **`expected_output`:**
```json
{
"answers": "In 2016 and 2017, Platt played in Dear Evan Hansen on Broadway at the Music Box Theatre.",
"doc_url": "http://en.wikipedia.org/wiki/Ben_Platt_(actor)",
"evidence": {
"table_array": [
["Year", "Production", "Role", "Venue", "Notes"],
["2002", "The Music Man", "Winthrop Paroo", "Hollywood Bowl", "Los Angeles"],
["2004", "Caroline, or Change", "Noah Gellman", "Ahmanson Theatre", "National tour"]
],
"highlighted_cell_ids": [[10, 1], [11, 0], [11, 1], [11, 3], [11, 4]]
}
}
```
*(The full `table_array` in the release contains all rows for the table; the snippet above is shortened for display.)*

**Example 2 — another question on the same document**

- **`input`:** `Which role did Ben Platt play at the Second Stage Theatre in 2016?`
- **`expected_output` (answers only):**
`"In 2016, Platt played the role of Evan Hansen at Second Stage Theatre."`
(Full JSON includes the same `doc_url` and `evidence` structure as in Example 1.)

**`metadata.value` (structure, Example 1):**
```json
{
"label_key": "Ben Platt (actor)",
"label_file": "feta_qa",
"doc_url": "http://en.wikipedia.org/wiki/Ben_Platt_(actor)",
"q_uid": "14717"
}
```

## References

### FeTa-QA (source task & data lineage)

Linyong Nan, Chiachun Hsieh, Ziming Mao, Xi Victoria Lin, Neha Verma, Rui Zhang, Wojciech Kryściński, Nick Schoelkopf, Riley Kong, Xiangru Tang, Murori Mutuma, Ben Rosand, Isabel Trindade, Renusree Bandaru, Jacob Cunningham, Caiming Xiong, Dragomir Radev. **FeTaQA: Free-form Table Question Answering.** *Transactions of the Association for Computational Linguistics*, 10:35–49, 2022.

- **Abstract (arXiv:2104.00369):** Existing table QA datasets emphasize factual questions with short answers; FeTa-QA introduces **10K** Wikipedia-based **{table, question, free-form answer, supporting table cells}** pairs, requiring **retrieval, inference, and integration** of multiple facts, with **human-generated** explanatory answers. The authors provide pipeline and end-to-end baselines and show FeTa-QA is challenging for both.
- **ACL Anthology (TACL):** [https://aclanthology.org/2022.tacl-1.3/](https://aclanthology.org/2022.tacl-1.3/)
- **arXiv:** [https://arxiv.org/abs/2104.00369](https://arxiv.org/abs/2104.00369)
- **Original project:** [Yale-LILY/FeTaQA](https://github.com/Yale-LILY/FeTaQA)

### UDA benchmark (suite containing this FeTa-QA slice)

Yulong Hui, Yao Lu, Huanchen Zhang. **UDA: A Benchmark Suite for Retrieval Augmented Generation in Real-world Document Analysis.** *NeurIPS 2024* (Datasets and Benchmarks Track).

- **Abstract (arXiv:2406.15187):** Introduces UDA with **2,965** real-world documents and **29,590** expert-annotated Q&A pairs across domains; evaluates LLM- and RAG-based document analysis and highlights **parsing** and **retrieval** design choices alongside answer quality.
- **arXiv:** [https://arxiv.org/abs/2406.15187](https://arxiv.org/abs/2406.15187)
- **Code & resources:** [https://github.com/qinchuanhui/UDA-Benchmark](https://github.com/qinchuanhui/UDA-Benchmark)

### Related Hub resources

- UDA QA aggregation (reference): [qinchuanhui/UDA-QA](https://huggingface.co/datasets/qinchuanhui/UDA-QA)

## Citation

If you use this dataset, please cite **FeTa-QA** and **UDA** (and this dataset record as appropriate):

```bibtex
@article{nan-etal-2022-fetaqa,
title = {FeTaQA: Free-form Table Question Answering},
author = {Nan, Linyong and Hsieh, Chiachun and Mao, Ziming and Lin, Xi Victoria and Verma, Neha and Zhang, Rui and Kry{\'s}ci{\'n}ski, Wojciech and Schoelkopf, Nick and Kong, Riley and Tang, Xiangru and Mutuma, Murori and Rosand, Ben and Trindade, Isabel and Bandaru, Renusree and Cunningham, Jacob and Xiong, Caiming and Radev, Dragomir},
journal = {Transactions of the Association for Computational Linguistics},
volume = {10},
year = {2022},
pages = {35--49}
}
```

```bibtex
@article{hui2024uda,
title = {UDA: A Benchmark Suite for Retrieval Augmented Generation in Real-world Document Analysis},
author = {Hui, Yulong and Lu, Yao and Zhang, Huanchen},
journal = {arXiv preprint arXiv:2406.15187},
year = {2024}
}
```

## License

This release is intended for research and evaluation. Use it in compliance with the **FeTa-QA** and **UDA** data licenses and terms (UDA’s Hugging Face distribution is documented as **CC BY-SA 4.0**; verify upstream conditions for your use case before redistribution or commercial use).