snorkelai/finqa-data

---
language:
- en
license: apache-2.0
task_categories:
- question-answering
tags:
- finance
- agents
- tool-use
- SEC-10K
- evaluation
size_categories:
- n<1K
configs:
- config_name: default
data_files:
- split: test
path: benchmark_questions/finqa.csv
---

# SnorkelFinance

Expert-verified financial QA dataset for evaluating AI agents on tool-calling and reasoning over SEC 10-K filings.

## Overview

SnorkelFinance is a benchmark of **290 questions** across **20 companies** spanning 5 industry verticals. Questions are created from 10-K filing documents and verified by Snorkel's network of financial experts on a 5-point scale for realism and accuracy.

Agents don't have direct access to the documents. Instead, they must plan and use provided tools (SQL queries, table lookups) to find and compute answers.

**Note:** This dataset is for evaluation only. Do not train on it.


## Data Structure

```
benchmark_questions/
finqa.csv # 290 evaluation questions
input_companies/
<company>/ # JSON/TXT table files extracted from SEC 10-K filings
tables_cleaned_all_companies.json # Table metadata
```

### CSV Columns

| Column | Description |
|--------|-------------|
| `id` | Unique question identifier |
| `user_query` | Full question prompt (includes company context) |
| `company` | Company name |
| `question` | The financial question |
| `answer` | Ground truth answer (in `\boxed{}` format) |

## Download

```bash
huggingface-cli download snorkelai/finqa-data --repo-type dataset --local-dir ./data
```

## Links

- [Leaderboard](https://snorkel.ai/leaderboard/category/snorkelfinance/) - Model scores and evaluation methodology
- [OpenEnv Environment](https://github.com/meta-pytorch/OpenEnv) - Runtime environment for running the benchmark