orgrctera/uda_fin_qa_qa

---
license: mit
language:
- en
pretty_name: UDA FinQA (Question Answering)
size_categories:
- 5K<n<10K
tags:
- finance
- question-answering
- numerical-reasoning
- unstructured-documents
- task:question-answering
configs:
- config_name: default
data_files:
- split: default
path: data/default-*
dataset_info:
features:
- name: input
dtype: string
- name: metadata
dtype: string
- name: answers
dtype: string
- name: evidence
dtype: string
- name: context
dtype: string
- name: program
dtype: string
splits:
- name: default
num_bytes: 43214536
num_examples: 8190
download_size: 6961204
dataset_size: 43214536
---

# UDA FinQA — Question Answering (`orgrctera/uda_fin_qa_qa`)

## Dataset description

This release is the **FinQA-aligned Question Answering (QA)** split from the **UDA (Unstructured Document Analysis)** benchmark: **8,190** expert-style question–answer instances grounded in real corporate financial disclosures (earnings materials, 10-K–style reports). Each row pairs a natural-language **question** with structured **supervision** (gold answers, supporting evidence, document context, and executable reasoning programs), so models can be trained or evaluated on **financial numerical reasoning** over heterogeneous evidence (narrative text and tables).

**UDA** (Hui et al., NeurIPS 2024) is a suite for **Retrieval-Augmented Generation (RAG)** and LLM-based analysis on **real-world** documents kept in original, messy formats. The finance track includes subsets such as **FinHybrid** (FinQA-style), designed to stress **parsing, alignment, and reasoning**—not only fluent generation.

**FinQA** (Chen et al., EMNLP 2021) is the foundational task: expert-written questions over financial reports with **multi-step numerical reasoning**, **heterogeneous evidence** (tables + text), and **explainable** annotations (including programs over quantities). This Hub dataset follows that task definition within UDA’s benchmark packaging (`sub_benchmark`: `fin_qa`).

## The task

- **Task type:** **Question Answering (QA)** for **FinQA** — given the question in `input`, predict the correct **numeric or textual answer** using information that would appear in the source report (tables and surrounding text). Evaluation typically uses gold **string** and **executable** answers and may use **evidence** and **program** consistency with the official FinQA / UDA protocols.
- **Input (`input`):** A single English question about reported figures, trends, or relationships (e.g., interest expense, growth rates, year-over-year comparisons).
- **Target (`expected_output`):** A JSON **string** with:
- **`answers`:** e.g. `str_answer` (normalized string) and `exe_answer` (numeric value where applicable).
- **`evidence`:** Pointers to supporting **text** and **table** snippets (e.g. `text_1`, `table_1`).
- **`context`:** Supporting **pre_text**, **post_text**, and **table** material aligned with the report excerpt.
- **`program`:** A **gold reasoning program** over numbers and operations (FinQA-style), supporting interpretability and program-based metrics.
- **Metadata (`metadata`):** `benchmark_name` (`uda_fin_qa`), `benchmark_type` (`uda`), `split`, `sub_benchmark` (`fin_qa`), and `value` (JSON with identifiers such as `label_key`, `label_file`, `q_uid`).

**Splits:** Single split `default` with **8,190** examples (one Parquet shard: `data/default-00000-of-00001.parquet`).

## Background

### FinQA

FinQA was introduced to study **numerical reasoning over financial data**: questions are written by finance professionals over real filings; annotations include operations and facts that support the answer. The authors show that strong general-domain LMs still **trail experts** on finance-specific knowledge and **multi-step** numeric reasoning. FinQA remains a standard benchmark for **table+text** reasoning in finance.

### UDA benchmark

UDA aggregates **thousands** of real documents and **tens of thousands** of annotated Q&A pairs across domains (including finance), with documents provided in ways that reflect **realistic** ingestion (e.g., PDF/HTML) so that **retrieval, chunking, and parsing** choices matter. The FinQA-related finance portion (FinHybrid / FinQA track) matches the scale of this dataset (**8,190** QA instances).

## References

### FinQA (source task and annotations)

**Zhiyu Chen, Wenhu Chen, Charese Smiley, Sameena Shah, Iana Borova, Dylan Langdon, Reema Moussa, Matt Beane, Ting-Hao Huang, Bryan Routledge, William Yang Wang.** *FinQA: A Dataset of Numerical Reasoning over Financial Data.* **EMNLP 2021**, pages 3697–3711.

- **Abstract (summary):** The paper introduces a large-scale dataset of QA pairs over financial reports with **gold reasoning programs** and evidence for explainability; experiments show pretrained language models **fall short of human experts** on financial knowledge and complex numerical reasoning, motivating better structured and unstructured reasoning over filings.

- **ACL Anthology:** [https://aclanthology.org/2021.emnlp-main.300/](https://aclanthology.org/2021.emnlp-main.300/)
- **DOI:** [10.18653/v1/2021.emnlp-main.300](https://doi.org/10.18653/v1/2021.emnlp-main.300)
- **Original code and data:** [https://github.com/czyssrs/FinQA](https://github.com/czyssrs/FinQA)
- **Project site:** [https://finqasite.github.io/](https://finqasite.github.io/)

### UDA (benchmark suite containing this FinQA slice)

**Yulong Hui, Yao Lu, Huanchen Zhang.** *UDA: A Benchmark Suite for Retrieval Augmented Generation in Real-world Document Analysis.* **NeurIPS 2024** (Datasets and Benchmarks Track).

- **Abstract (summary):** UDA provides a benchmark for **RAG** in **real-world document analysis** with diverse domains and question types, using real documents and expert annotations; the work analyzes how **parsing and retrieval** interact with generation and highlights practical design choices for document AI systems.

- **arXiv:** [https://arxiv.org/abs/2406.15187](https://arxiv.org/abs/2406.15187)
- **NeurIPS proceedings:** [https://proceedings.neurips.cc/paper_files/paper/2024/hash/7c06759d1a8567f087b02e8589454917-Abstract-Datasets_and_Benchmarks_Track.html](https://proceedings.neurips.cc/paper_files/paper/2024/hash/7c06759d1a8567f087b02e8589454917-Abstract-Datasets_and_Benchmarks_Track.html)
- **Code:** [https://github.com/qinchuanhui/UDA-Benchmark](https://github.com/qinchuanhui/UDA-Benchmark)

### Related Hub resources

- Aggregated UDA QA reference: [qinchuanhui/UDA-QA](https://huggingface.co/datasets/qinchuanhui/UDA-QA)

## Examples

Illustrative rows from the dataset; long `context` blocks are abbreviated.

### Example 1 — interest expense

**`input`:**

```text
what is the the interest expense in 2009?
```

**`expected_output` (excerpt; `context` shortened):**

```json
{
"answers": {
"str_answer": "380",
"exe_answer": 3.8
},
"evidence": {
"text_1": "if libor changes by 100 basis points , our annual interest expense would change by $ 3.8 million ."
},
"context": {
"pre_text": [
"interest rate to a variable interest rate based on the three-month libor plus 2.05% ...",
"if libor changes by 100 basis points , our annual interest expense would change by $ 3.8 million .",
"..."
],
"post_text": ["..."]
},
"program": "divide(100, 100), divide(3.8, #0)"
}
```

### Example 2 — amortization growth

**`input`:**

```text
what is the expected growth rate in amortization expense in 2010?
```

**`expected_output` (excerpt):**

```json
{
"answers": {
"str_answer": "-27.0%",
"exe_answer": -0.26689
},
"evidence": {
"table_1": "fiscal years the 2010 of amortization expense is $ 5425 ;",
"text_2": "amortization expense from continuing operations , related to intangibles was $ 7.4 million , $ 9.3 million and $ 9.2 million in fiscal 2009 , 2008 and 2007 , respectively ."
},
"context": { "...": "..." },
"program": "subtract(1074.5, 1110.6), divide(#0, 1110.6)"
}
```

**`metadata.value` (example):**

```json
{
"label_key": "ADI_2009",
"label_file": "fin_qa",
"q_uid": "ADI/2009/page_49.pdf-1"
}
```

## Citation

If you use this dataset, please cite **FinQA**, **UDA**, and this dataset record as appropriate:

```bibtex
@inproceedings{chen-etal-2021-finqa,
title = {FinQA: A Dataset of Numerical Reasoning over Financial Data},
author = {Chen, Zhiyu and Chen, Wenhu and Smiley, Charese and Shah, Sameena and Borova, Iana and Langdon, Dylan and Moussa, Reema and Beane, Matt and Huang, Ting-Hao and Routledge, Bryan and Wang, William Yang},
booktitle = {Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing},
year = {2021},
pages = {3697--3711}
}
```

```bibtex
@inproceedings{hui2024uda,
title = {UDA: A Benchmark Suite for Retrieval Augmented Generation in Real-world Document Analysis},
author = {Hui, Yulong and Lu, Yao and Zhang, Huanchen},
booktitle = {Advances in Neural Information Processing Systems},
year = {2024}
}
```

## License

The original FinQA release is under the **MIT License** (see the [FinQA repository](https://github.com/czyssrs/FinQA)). Use this dataset in compliance with the original data licenses and the UDA benchmark terms.

---
许可证：MIT
语言：
- 英语
规范名称：UDA FinQA（问答任务）
样本规模类别：
- 5000 < 样本数 < 10000
标签：
- 金融
- 问答
- 数值推理
- 非结构化文档
- 任务：问答
配置：
- 配置名称：default
数据文件：
- 拆分：default
路径：data/default-*
数据集信息：
特征：
- 名称：input
数据类型：字符串
- 名称：metadata
数据类型：字符串
- 名称：answers
数据类型：字符串
- 名称：evidence
数据类型：字符串
- 名称：context
数据类型：字符串
- 名称：program
数据类型：字符串
拆分：
- 名称：default
字节数：43214536
样本数：8190
下载大小：6961204
数据集总大小：43214536
---

# UDA FinQA — 问答任务（`orgrctera/uda_fin_qa_qa`）

## 数据集描述

本版本是**非结构化文档分析（Unstructured Document Analysis, UDA）**基准测试中与**FinQA对齐的问答（QA）**拆分集：包含**8190**个基于真实企业财务披露（收益材料、10-K类报告）构建的专家风格问答实例。每一行均将自然语言**问题**与结构化**监督数据**（标准答案、支撑证据、文档上下文、可执行推理程序）配对，使得模型可针对异构证据（叙述性文本与表格）开展**金融数值推理**的训练与评估。

**UDA**（Hui等人，NeurIPS 2024）是一套面向**真实世界**文档的**检索增强生成（Retrieval-Augmented Generation, RAG）**与基于大语言模型（Large Language Model, LLM）分析的工具套件，文档以原始、非规整格式提供。其金融赛道包含如**FinHybrid**（FinQA风格）等子集，旨在对**解析、对齐与推理**能力进行严苛测试，而非仅考量流畅生成能力。

**FinQA**（Chen等人，EMNLP 2021）是基础任务：由专业人员基于财务报告编写的问题，需完成**多步数值推理**，依赖**异构证据**（表格+文本），并带有**可解释**标注（包括针对数量的推理程序）。本Hugging Face数据集遵循UDA基准包装中的该任务定义（`sub_benchmark`: `fin_qa`）。

## 任务

- **任务类型**：面向FinQA的**问答任务** — 给定`input`中的问题，利用源报告中出现的信息（表格及周边文本）预测正确的**数值或文本答案**。评估通常使用标准答案字符串与可执行答案，也可依据官方FinQA/UDA协议，通过**证据**与**推理程序**的一致性进行评测。
- **输入（`input`）**：针对报告中的数据、趋势或关联关系的单条英语问题（例如利息支出、增长率、同比对比等）。
- **目标（`expected_output`）**：包含以下内容的JSON字符串：
- **`answers`**：例如`str_answer`（归一化字符串答案）与`exe_answer`（适用时的数值答案）。
- **`evidence`**：指向支撑性**文本**与**表格**片段的指针（例如`text_1`、`table_1`）。
- **`context`**：与报告节选对齐的支撑性**前置文本**、**后置文本**与**表格**材料。
- **`program`**：基于数值与运算的**标准答案推理程序**（FinQA风格），支持可解释性与基于程序的评估指标。
- **元数据（`metadata`）**：包含`benchmark_name`（`uda_fin_qa`）、`benchmark_type`（`uda`）、拆分信息、`sub_benchmark`（`fin_qa`）以及`value`（包含`label_key`、`label_file`、`q_uid`等标识符的JSON对象）。

**拆分**：仅包含`default`拆分，共**8190**个样本（一个Parquet分片：`data/default-00000-of-00001.parquet`）。

## 背景

### FinQA
FinQA旨在研究**金融数据的数值推理**：问题由金融专业人员基于真实备案文件编写，标注包含支撑答案的运算步骤与事实。相关研究表明，优秀的通用领域大语言模型在金融领域专有知识与**多步**数值推理任务上仍**落后于人类专家**。FinQA仍是金融领域**表格+文本**推理的标准基准测试。

### UDA基准测试套件
UDA整合了跨多个领域（包含金融）的**数千份**真实文档与**数万个**标注问答对，文档以贴合真实摄取场景的格式提供（例如PDF/HTML），因此**检索、分块与解析**的选择会对结果产生影响。与FinQA相关的金融赛道部分（FinHybrid/FinQA赛道）与本数据集规模相当（**8190**个问答实例）。

## 参考文献

### FinQA（源任务与标注）
**Zhiyu Chen, Wenhu Chen, Charese Smiley, Sameena Shah, Iana Borova, Dylan Langdon, Reema Moussa, Matt Beane, Ting-Hao Huang, Bryan Routledge, William Yang Wang.** *FinQA: A Dataset of Numerical Reasoning over Financial Data.* **EMNLP 2021**, 第3697–3711页。

- **摘要**：本文提出了一个大规模的问答对数据集，基于包含**标准答案推理程序**与可解释性证据的财务报告构建；实验表明，预训练语言模型在金融知识与复杂数值推理任务上**仍不及人类专家**，这推动了针对备案文件的结构化与非结构化推理研究的发展。
- **ACL Anthology链接**：[https://aclanthology.org/2021.emnlp-main.300/](https://aclanthology.org/2021.emnlp-main.300/)
- **DOI**：[10.18653/v1/2021.emnlp-main.300](https://doi.org/10.18653/v1/2021.emnlp-main.300)
- **原始代码与数据**：[https://github.com/czyssrs/FinQA](https://github.com/czyssrs/FinQA)
- **项目主页**：[https://finqasite.github.io/](https://finqasite.github.io/)

### UDA（包含本FinQA子集的基准测试套件）
**Yulong Hui, Yao Lu, Huanchen Zhang.** *UDA: A Benchmark Suite for Retrieval Augmented Generation in Real-world Document Analysis.* **NeurIPS 2024**（数据集与基准测试赛道）。

- **摘要**：UDA为真实世界文档分析中的**RAG**提供基准测试，涵盖多样领域与问题类型，使用真实文档与专家标注；该工作分析了**解析与检索**如何与生成交互，并为文档AI系统的实际设计选择提供了参考。
- **arXiv链接**：[https://arxiv.org/abs/2406.15187](https://arxiv.org/abs/2406.15187)
- **NeurIPS会议论文集链接**：[https://proceedings.neurips.cc/paper_files/paper/2024/hash/7c06759d1a8567f087b02e8589454917-Abstract-Datasets_and_Benchmarks_Track.html](https://proceedings.neurips.cc/paper_files/paper/2024/hash/7c06759d1a8567f087b02e8589454917-Abstract-Datasets_and_Benchmarks_Track.html)
- **代码仓库**：[https://github.com/qinchuanhui/UDA-Benchmark](https://github.com/qinchuanhui/UDA-Benchmark)

### 相关Hugging Face Hub资源
- 聚合UDA问答参考数据集：[qinchuanhui/UDA-QA](https://huggingface.co/datasets/qinchuanhui/UDA-QA)

## 示例

本数据集的示意性样本行；较长的`context`块已进行缩写。

### 示例1 — 利息支出
**`input`:**
text
what is the the interest expense in 2009?

**`expected_output`（节选；`context`已缩写）:**
json
{
"answers": {
"str_answer": "380",
"exe_answer": 3.8
},
"evidence": {
"text_1": "if libor changes by 100 basis points , our annual interest expense would change by $ 3.8 million ."
},
"context": {
"pre_text": [
"interest rate to a variable interest rate based on the three-month libor plus 2.05% ...",
"if libor changes by 100 basis points , our annual interest expense would change by $ 3.8 million .",
"..."
],
"post_text": ["..."]
},
"program": "divide(100, 100), divide(3.8, #0)"
}

### 示例2 — 摊销费用增长率
**`input`:**
text
what is the expected growth rate in amortization expense in 2010?

**`expected_output`（节选）:**
json
{
"answers": {
"str_answer": "-27.0%",
"exe_answer": -0.26689
},
"evidence": {
"table_1": "fiscal years the 2010 of amortization expense is $ 5425 ;",
"text_2": "amortization expense from continuing operations , related to intangibles was $ 7.4 million , $ 9.3 million and $ 9.2 million in fiscal 2009 , 2008 and 2007 , respectively ."
},
"context": { "...": "..." },
"program": "subtract(1074.5, 1110.6), divide(#0, 1110.6)"
}

**`metadata.value`（示例）:**
json
{
"label_key": "ADI_2009",
"label_file": "fin_qa",
"q_uid": "ADI/2009/page_49.pdf-1"
}

## 引用

若使用本数据集，请酌情引用**FinQA**、**UDA**以及本数据集记录：
bibtex
@inproceedings{chen-etal-2021-finqa,
title = {FinQA: A Dataset of Numerical Reasoning over Financial Data},
author = {Chen, Zhiyu and Chen, Wenhu and Smiley, Charese and Shah, Sameena and Borova, Iana and Langdon, Dylan and Moussa, Reema and Beane, Matt and Huang, Ting-Hao and Routledge, Bryan and Wang, William Yang},
booktitle = {Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing},
year = {2021},
pages = {3697--3711}
}

bibtex
@inproceedings{hui2024uda,
title = {UDA: A Benchmark Suite for Retrieval Augmented Generation in Real-world Document Analysis},
author = {Hui, Yulong and Lu, Yao and Zhang, Huanchen},
booktitle = {Advances in Neural Information Processing Systems},
year = {2024}
}

## 许可证

原始FinQA版本遵循**MIT许可证**（详见[FinQA代码仓库](https://github.com/czyssrs/FinQA)）。使用本数据集时，请遵守原始数据许可证与UDA基准测试的使用条款。