cetusian/markdown-table-qa-09

---
dataset_info:
features:
- name: id
dtype: string
- name: instruction
dtype: string
- name: input
dtype: string
- name: response
dtype: string
- name: domain
dtype: string
- name: question_type
dtype: string
- name: n_rows
dtype: int64
- name: n_cols
dtype: int64
- name: numeric_cols
list: string
- name: categorical_cols
list: string
splits:
- name: train
num_examples: 2000
- name: validation
num_examples: 200
configs:
- config_name: default
data_files:
- split: train
path: data/train-*
- split: validation
path: data/validation-*
---
# Markdown Table QA Dataset — Part 09/20

Part **9** of a 20-dataset collection for training and evaluating language models on structured table understanding and computational reasoning. Each part contains **2,200 samples** (2,000 train + 200 validation) with step-by-step reasoning traces.

See the full collection: [cetusian/markdown-table-qa-01](https://huggingface.co/datasets/cetusian/markdown-table-qa-01) through [cetusian/markdown-table-qa-20](https://huggingface.co/datasets/cetusian/markdown-table-qa-20)

Parent dataset: [cetusian/markdown-table-qa](https://huggingface.co/datasets/cetusian/markdown-table-qa) (11,000 samples)

---

## What's in it

Each sample contains a markdown table paired with a natural language question and a detailed answer with step-by-step reasoning:

| Field | Description |
|---|---|
| `instruction` | Natural language question about the table |
| `input` | The markdown table |
| `response` | Answer with `<think>...</think>` reasoning trace followed by a final answer |
| `domain` | Table domain (e.g. `healthcare_appointments`, `wildlife_survey`) |
| `question_type` | One of 12 types — equally balanced (~167 train + ~17 val per type) |

### Reasoning format

Every response includes a detailed `<think>` block that:
- Quotes **exact cell values** from the table
- Shows **all arithmetic step by step** (`a + b = c; c + d = e`)
- Enumerates rows explicitly by name for counting tasks
- Never skips to final results

---

## Question types (equally balanced)

| Type | Description |
|---|---|
| `sum` | Sum a numeric column |
| `mean` | Average of a numeric column |
| `max_row` | Row with highest value |
| `min_row` | Row with lowest value |
| `filtered_sum` | Sum with a filter condition |
| `filtered_count` | Count with a filter condition |
| `percentage` | Percentage of rows matching a condition |
| `rank_top3` | Top 3 rows by a numeric column |
| `comparison` | Compare values between two rows |
| `lookup` | Look up a specific cell value |
| `compound` | Multi-part question combining lookups |
| `summarization` | Summarize the entire table |

Computational types have **mathematically verified answers** computed with pandas.

---

## Domains

35 real-world domains covering diverse table structures including healthcare, finance, sports, e-commerce, energy, wildlife, logistics, and more.

---

## How to use

```python
from datasets import load_dataset

ds = load_dataset("cetusian/markdown-table-qa-09")

# Load all 20 parts
from datasets import concatenate_datasets
all_train = concatenate_datasets([
load_dataset(f"cetusian/markdown-table-qa-{i:02d}", split="train")
for i in range(1, 21)
])
# -> 40,000 training samples
```

---

## Generation

Generated using a pipeline built on **[vLLM](https://github.com/vllm-project/vllm)** with **OpenAI gpt-oss-120b** (4 GPUs, tensor parallelism). Quality-filtered for proper reasoning traces, answer grounding, and balanced type distribution.

---

## About Surogate

**[Surogate](https://surogate.ai)** is a full-stack AgentOps platform for developing, deploying, evaluating, and monitoring reliable AI agents — built by [Invergent AI](https://github.com/invergent-ai/surogate).

数据集信息：
特征字段：
- 名称：id，数据类型：字符串（string）
- 名称：instruction，数据类型：字符串
- 名称：input，数据类型：字符串
- 名称：response，数据类型：字符串
- 名称：domain，数据类型：字符串
- 名称：question_type，数据类型：字符串
- 名称：n_rows，数据类型：64位整型（int64）
- 名称：n_cols，数据类型：64位整型
- 名称：numeric_cols，列表类型，元素为字符串
- 名称：categorical_cols，列表类型，元素为字符串
数据划分：
- 划分名称：train，样本数量：2000
- 划分名称：validation，样本数量：200
配置项：
- 配置名称：default，数据文件：
- 划分：train，路径：data/train-*
- 划分：validation，路径：data/validation-*

# Markdown表格问答数据集 — 第09/20部分

本数据集为20个数据集合集的第9部分，专为训练和评估语言模型的结构化表格理解与计算推理能力而设计。每个子数据集均包含2200条样本（2000条训练样本+200条验证样本），并附带逐步推理轨迹。

完整合集可从[cetusian/markdown-table-qa-01](https://huggingface.co/datasets/cetusian/markdown-table-qa-01)至[cetusian/markdown-table-qa-20](https://huggingface.co/datasets/cetusian/markdown-table-qa-20)获取。

父数据集：[cetusian/markdown-table-qa](https://huggingface.co/datasets/cetusian/markdown-table-qa)（共11000条样本）

## 数据集内容说明

每条样本均包含一份Markdown格式表格、配套的自然语言问题，以及带有逐步推理过程的详细答案：

| 字段 | 说明 |
|---|---|
| `instruction` | 针对表格的自然语言查询问题 |
| `input` | Markdown格式的原始表格 |
| `response` | 答案内容，格式为`<think>...</think>`推理轨迹后接最终答案 |
| `domain` | 表格所属领域（例如`healthcare_appointments`、`wildlife_survey`） |
| `question_type` | 共12种题型，分布均衡（每种题型约含167条训练样本+17条验证样本） |

### 推理格式规范

所有答案均包含详细的`<think>`推理块，需满足以下要求：
- 引用表格中的**精确单元格值**
- 展示**完整的分步算术运算过程**（例如`a + b = c; c + d = e`）
- 针对计数任务明确枚举行名称
- 绝不跳过步骤直接给出最终结果

## 题型分布（均衡配置）

| 题型代码 | 题型说明 |
|---|---|
| `sum` | 对指定数值列求和 |
| `mean` | 计算指定数值列的平均值 |
| `max_row` | 定位数值最高的行 |
| `min_row` | 定位数值最低的行 |
| `filtered_sum` | 带筛选条件的求和任务 |
| `filtered_count` | 带筛选条件的计数任务 |
| `percentage` | 计算符合指定条件的行占总样本的百分比 |
| `rank_top3` | 按指定数值列排序的前3行 |
| `comparison` | 对比两行之间的数值差异 |
| `lookup` | 查询指定单元格的数值 |
| `compound` | 结合多项查询逻辑的复合问题 |
| `summarization` | 对整个表格内容进行总结 |

计算类题型的答案均经过**基于pandas的数学验证**，确保结果准确。

## 覆盖领域

本数据集涵盖35个真实世界领域，包含多样化的表格结构，涉及医疗、金融、体育、电商、能源、野生动物、物流等多个场景。

## 使用方法

python
from datasets import load_dataset

ds = load_dataset("cetusian/markdown-table-qa-09")

# 加载全部20个子数据集
from datasets import concatenate_datasets
all_train = concatenate_datasets([
load_dataset(f"cetusian/markdown-table-qa-{i:02d}", split="train")
for i in range(1, 21)
])
# -> 总计40000条训练样本

## 数据集生成

本数据集基于**[vLLM](https://github.com/vllm-project/vllm)**搭建的流水线生成，搭配**OpenAI gpt-oss-120b**模型（使用4张GPU进行张量并行计算）。生成后经过严格质量过滤，确保推理轨迹规范、答案锚定表格原文且题型分布均衡。

## 关于Surogate

**[Surogate](https://surogate.ai)** 是一款全栈式AI智能体（AI Agent）运维平台，用于开发、部署、评估和监控可靠的AI智能体 —— 由[Invergent AI](https://github.com/invergent-ai/surogate)开发。