cetusian/markdown-table-qa-15

---
dataset_info:
features:
- name: id
dtype: string
- name: instruction
dtype: string
- name: input
dtype: string
- name: response
dtype: string
- name: domain
dtype: string
- name: question_type
dtype: string
- name: n_rows
dtype: int64
- name: n_cols
dtype: int64
- name: numeric_cols
list: string
- name: categorical_cols
list: string
splits:
- name: train
num_examples: 2000
- name: validation
num_examples: 200
configs:
- config_name: default
data_files:
- split: train
path: data/train-*
- split: validation
path: data/validation-*
---
# Markdown Table QA Dataset — Part 15/20

Part **15** of a 20-dataset collection for training and evaluating language models on structured table understanding and computational reasoning. Each part contains **2,200 samples** (2,000 train + 200 validation) with step-by-step reasoning traces.

See the full collection: [cetusian/markdown-table-qa-01](https://huggingface.co/datasets/cetusian/markdown-table-qa-01) through [cetusian/markdown-table-qa-20](https://huggingface.co/datasets/cetusian/markdown-table-qa-20)

Parent dataset: [cetusian/markdown-table-qa](https://huggingface.co/datasets/cetusian/markdown-table-qa) (11,000 samples)

---

## What's in it

Each sample contains a markdown table paired with a natural language question and a detailed answer with step-by-step reasoning:

| Field | Description |
|---|---|
| `instruction` | Natural language question about the table |
| `input` | The markdown table |
| `response` | Answer with `<think>...</think>` reasoning trace followed by a final answer |
| `domain` | Table domain (e.g. `healthcare_appointments`, `wildlife_survey`) |
| `question_type` | One of 12 types — equally balanced (~167 train + ~17 val per type) |

### Reasoning format

Every response includes a detailed `<think>` block that:
- Quotes **exact cell values** from the table
- Shows **all arithmetic step by step** (`a + b = c; c + d = e`)
- Enumerates rows explicitly by name for counting tasks
- Never skips to final results

---

## Question types (equally balanced)

| Type | Description |
|---|---|
| `sum` | Sum a numeric column |
| `mean` | Average of a numeric column |
| `max_row` | Row with highest value |
| `min_row` | Row with lowest value |
| `filtered_sum` | Sum with a filter condition |
| `filtered_count` | Count with a filter condition |
| `percentage` | Percentage of rows matching a condition |
| `rank_top3` | Top 3 rows by a numeric column |
| `comparison` | Compare values between two rows |
| `lookup` | Look up a specific cell value |
| `compound` | Multi-part question combining lookups |
| `summarization` | Summarize the entire table |

Computational types have **mathematically verified answers** computed with pandas.

---

## Domains

35 real-world domains covering diverse table structures including healthcare, finance, sports, e-commerce, energy, wildlife, logistics, and more.

---

## How to use

```python
from datasets import load_dataset

ds = load_dataset("cetusian/markdown-table-qa-15")

# Load all 20 parts
from datasets import concatenate_datasets
all_train = concatenate_datasets([
load_dataset(f"cetusian/markdown-table-qa-{i:02d}", split="train")
for i in range(1, 21)
])
# -> 40,000 training samples
```

---

## Generation

Generated using a pipeline built on **[vLLM](https://github.com/vllm-project/vllm)** with **OpenAI gpt-oss-120b** (4 GPUs, tensor parallelism). Quality-filtered for proper reasoning traces, answer grounding, and balanced type distribution.

---

## About Surogate

**[Surogate](https://surogate.ai)** is a full-stack AgentOps platform for developing, deploying, evaluating, and monitoring reliable AI agents — built by [Invergent AI](https://github.com/invergent-ai/surogate).

数据集信息：
特征字段：
- 字段名：id，数据类型：字符串型
- 字段名：instruction，数据类型：字符串型
- 字段名：input，数据类型：字符串型
- 字段名：response，数据类型：字符串型
- 字段名：domain，数据类型：字符串型
- 字段名：question_type，数据类型：字符串型
- 字段名：n_rows，数据类型：64位整型
- 字段名：n_cols，数据类型：64位整型
- 字段名：numeric_cols，数据类型：字符串列表
- 字段名：categorical_cols，数据类型：字符串列表
划分集：
- 划分名称：train，样本数量：2000
- 划分名称：validation，样本数量：200
配置项：
- 配置名称：default，数据文件：
- 划分集：train，路径：data/train-*
- 划分集：validation，路径：data/validation-*

# Markdown表格问答数据集 — 第15/20部分

本数据集为20个数据集合集的第15部分，用于训练和评估大语言模型的结构化表格理解与计算推理能力。每个部分包含2200个样本（2000个训练样本 + 200个验证样本），并附带逐步推理轨迹。

完整合集可访问：[cetusian/markdown-table-qa-01](https://huggingface.co/datasets/cetusian/markdown-table-qa-01) 至 [cetusian/markdown-table-qa-20](https://huggingface.co/datasets/cetusian/markdown-table-qa-20)

父级数据集：[cetusian/markdown-table-qa](https://huggingface.co/datasets/cetusian/markdown-table-qa)，共11000个样本

---

## 数据集内容

每个样本包含一个Markdown表格、配套的自然语言问题，以及附带逐步推理轨迹的详细答案：

| 字段名 | 描述 |
|---|---|
| `instruction` | 针对表格的自然语言问题 |
| `input` | Markdown表格本体 |
| `response` | 包含`<think>...</think>`推理轨迹，后接最终答案的完整回复 |
| `domain` | 表格所属领域（例如`healthcare_appointments`、`wildlife_survey`） |
| `question_type` | 共12种题型，分布均衡（每种题型约含167个训练样本、17个验证样本） |

### 推理格式

所有回复均包含详细的`<think>`推理块，该块需满足：
- 引用表格中**精确的单元格值**
- 逐步展示所有计算过程（如`a + b = c; c + d = e`）
- 针对计数类任务，明确按名称枚举所涉及的行
- 绝不省略中间步骤直接给出最终结果

---

## 均衡分布的题型

| 题型代号 | 题型描述 |
|---|---|
| `sum` | 对数值列求和 |
| `mean` | 计算数值列的平均值 |
| `max_row` | 取值最高的行 |
| `min_row` | 取值最低的行 |
| `filtered_sum` | 带筛选条件的求和 |
| `filtered_count` | 带筛选条件的计数 |
| `percentage` | 符合指定条件的行占比 |
| `rank_top3` | 按数值列排序的前3行 |
| `comparison` | 比较两行的数值 |
| `lookup` | 查询指定单元格的值 |
| `compound` | 结合多项查询的复合问题 |
| `summarization` | 对整张表格进行总结 |

所有计算类题型的答案均经**数学验证**，计算过程基于pandas实现。

---

## 覆盖领域

涵盖35个真实世界领域，包含多种表格结构类型，涉及医疗、金融、体育、电子商务、能源、野生动物、物流等多个领域。

---

## 使用方法

python
from datasets import load_dataset

# 加载第15部分数据集
ds = load_dataset("cetusian/markdown-table-qa-15")

# 加载全部20个部分的训练集
from datasets import concatenate_datasets
all_train = concatenate_datasets([
load_dataset(f"cetusian/markdown-table-qa-{i:02d}", split="train")
for i in range(1, 21)
])
# 总计可获得40000个训练样本

---

## 数据集生成

本数据集基于**[vLLM](https://github.com/vllm-project/vllm)**构建的流水线生成，使用**OpenAI gpt-oss-120b**模型（依托4张GPU实现张量并行）。生成后经过质量过滤，确保推理轨迹规范、答案锚定准确且题型分布均衡。

---

## 关于Surogate

**[Surogate](https://surogate.ai)**是一款全栈式AgentOps平台，用于开发、部署、评估与监控可靠的AI智能体，由[Invergent AI](https://github.com/invergent-ai/surogate)开发。