cetusian/markdown-table-expert

---
dataset_info:
features:
- name: id
dtype: string
- name: instruction
dtype: string
- name: input
dtype: string
- name: response
dtype: string
- name: domain
dtype: string
- name: question_type
dtype: string
- name: n_rows
dtype: int64
- name: n_cols
dtype: int64
- name: numeric_cols
list: string
- name: categorical_cols
list: string
splits:
- name: train
num_examples: 40000
- name: validation
num_examples: 4000
configs:
- config_name: default
data_files:
- split: train
path: data/train-*
- split: validation
path: data/validation-*
license: apache-2.0
task_categories:
- question-answering
- table-question-answering
language:
- en
tags:
- markdown
- tables
- reasoning
- chain-of-thought
- structured-data
- fine-tuning
size_categories:
- 10K<n<100K
---

# Markdown Table Expert

A large-scale dataset for teaching language models to **read, understand, and reason over markdown tables**. Contains **44,000 samples** (40,000 train + 4,000 validation) spanning 35 real-world domains with detailed step-by-step reasoning traces.

## Why This Dataset

Markdown tables are everywhere — in documentation, reports, READMEs, financial statements, and web content. Yet most LLMs struggle with structured tabular data, especially when asked to perform calculations, comparisons, or multi-step reasoning over cells and rows.

This dataset teaches models to:

1. **Parse markdown table structure** — correctly identify rows, columns, headers, and cell values
2. **Extract precise values** — look up specific cells without hallucinating or paraphrasing
3. **Perform arithmetic** — sum, average, count, and compute percentages with step-by-step work shown
4. **Reason across rows** — compare entries, rank values, filter by conditions, and synthesize information
5. **Show their work** — every answer includes a detailed `<think>` reasoning trace before the final answer

## Dataset Format

Each sample contains:

| Field | Description |
|---|---|
| `instruction` | A natural language question about the table |
| `input` | A markdown-formatted table |
| `response` | A `<think>...</think>` reasoning trace followed by the final answer |
| `domain` | The real-world domain of the table (e.g., `healthcare_appointments`, `energy_consumption`) |
| `question_type` | One of 12 question types (see below) |
| `n_rows` / `n_cols` | Table dimensions |
| `numeric_cols` / `categorical_cols` | Column type metadata |

### Example

```
instruction: "What is the total revenue across all stores?"
input: "| Store | Revenue | Employees |\n|---|---|---|\n| Downtown | 45000 | 12 |\n| Mall | 62000 | 18 |\n| Airport | 38000 | 8 |"
response: "<think>\nI need to sum the Revenue column.\n\nDowntown: 45000\nMall: 62000\nAirport: 38000\n\n45000 + 62000 = 107000\n107000 + 38000 = 145000\n</think>\n\nThe total revenue across all stores is **$145,000**."
```

## Question Types

The dataset is **equally balanced** across 12 question types (~3,333 train / ~333 val each):

| Type | Description | Answer Verification |
|---|---|---|
| `sum` | Sum a numeric column | Computed with pandas |
| `mean` | Average of a numeric column | Computed with pandas |
| `max_row` | Row with the highest value | Computed with pandas |
| `min_row` | Row with the lowest value | Computed with pandas |
| `filtered_sum` | Sum with a filter condition | Computed with pandas |
| `filtered_count` | Count with a filter condition | Computed with pandas |
| `percentage` | Percentage of rows matching a condition | Computed with pandas |
| `rank_top3` | Top 3 rows by a numeric column | Computed with pandas |
| `comparison` | Compare values between rows | LLM-generated |
| `lookup` | Look up a specific cell value | LLM-generated |
| `compound` | Multi-part question combining operations | LLM-generated |
| `summarization` | Summarize the entire table | LLM-generated |

**Computational types** (sum through rank_top3) have **mathematically verified answers** — the ground truth is computed with pandas, and the LLM only generates the reasoning trace to explain the calculation. This guarantees numerical correctness.

## Reasoning Trace Quality

Every `<think>` block in this dataset follows strict quality standards:

- **Exact cell quoting** — values are copied verbatim from the table, never paraphrased
- **Step-by-step arithmetic** — calculations are shown incrementally (`a + b = X; X + c = Y`), never jumping to final results
- **Explicit row enumeration** — counting tasks list rows by name rather than claiming totals
- **Data rows only** — percentage and count calculations correctly exclude the header row

Samples with lazy, placeholder, or incomplete reasoning traces are filtered out during generation.

## Domains

35 diverse real-world domains ensure the model generalizes across different table structures and vocabularies:

`healthcare_appointments` · `social_media_campaign` · `employee_hr` · `ecommerce_products` · `student_grades` · `project_tracking` · `retail_store_performance` · `financial_transactions` · `sports_team_stats` · `inventory_management` · `customer_support` · `marketing_leads` · `event_registration` · `restaurant_menu` · `flight_operations` · `weather_station` · `real_estate_listings` · `movie_box_office` · `hospital_lab_results` · `energy_consumption` · `gym_membership` · `agriculture_yield` · `shipping_logistics` · `music_streaming` · `vehicle_fleet` · `online_courses` · `call_center` · `pollution_monitoring` · `hotel_bookings` · `manufacturing_qc` · `library_circulation` · `charity_donations` · `app_store_analytics` · `public_transit` · `wildlife_survey`

## Usage

```python
from datasets import load_dataset

ds = load_dataset("cetusian/markdown-table-expert")

# Training split
train = ds["train"] # 40,000 samples
val = ds["validation"] # 4,000 samples

# Filter by question type
math_samples = train.filter(lambda x: x["question_type"] in ["sum", "mean", "percentage"])

# Filter by domain
healthcare = train.filter(lambda x: x["domain"] == "healthcare_appointments")
```

### Fine-tuning

The dataset is designed for supervised fine-tuning (SFT) of language models. The `<think>` tags can be used as-is for chain-of-thought training, or stripped if your training framework adds them automatically.

```python
# Format for chat-style fine-tuning
def format_sample(sample):
return {
"messages": [
{"role": "user", "content": f"{sample['instruction']}\n\n{sample['input']}"},
{"role": "assistant", "content": sample["response"]},
]
}
```

## Statistics

| Split | Samples | Domains | Question Types |
|---|---|---|---|
| Train | 40,000 | 35 | 12 (balanced) |
| Validation | 4,000 | 35 | 12 (balanced) |
| **Total** | **44,000** | **35** | **12** |

## License

Apache 2.0

dataset_info: 数据集信息
features: 特征字段
- 字段名：id，数据类型：字符串
- 字段名：instruction，数据类型：字符串
- 字段名：input，数据类型：字符串
- 字段名：response，数据类型：字符串
- 字段名：domain，数据类型：字符串
- 字段名：question_type，数据类型：字符串
- 字段名：n_rows，数据类型：64位整型
- 字段名：n_cols，数据类型：64位整型
- 字段名：numeric_cols，数据类型：字符串列表
- 字段名：categorical_cols，数据类型：字符串列表
splits: 数据集划分
- 划分名称：train（训练集），样本数：40000
- 划分名称：validation（验证集），样本数：4000
configs: 配置项
- 配置名称：default（默认配置），数据文件：
- 训练集划分：路径为data/train-*
- 验证集划分：路径为data/validation-*
license: 许可证：Apache 2.0
task_categories: 任务类别：问答、表格问答
language: 语言：英语
tags: 标签：Markdown、表格、推理、思维链（Chain-of-Thought）、结构化数据、微调
size_categories: 样本规模：10000 < n < 100000

# Markdown 表格专家（Markdown Table Expert）

这是一个用于教授大语言模型（Large Language Model）**读取、理解并对Markdown表格进行推理**的大规模数据集，包含44000条样本（40000条训练样本 + 4000条验证样本），覆盖35个真实场景领域，且每条样本均附带详细的分步推理轨迹。

## 数据集构建动机

Markdown表格广泛应用于文档、报告、README文件、财务报表及网页内容中，但当前多数大语言模型在处理结构化表格数据时表现欠佳，尤其是在需要对单元格、行进行计算、比较或多步推理的场景下。本数据集旨在帮助模型掌握以下能力：
1. **解析Markdown表格结构**：准确识别行、列、表头及单元格值
2. **精准提取数值**：精准查找指定单元格内容，避免幻觉或释义偏差
3. **执行算术运算**：支持求和、求平均、计数及百分比计算，并附带分步推导过程
4. **跨行推理**：对条目进行比较、值排序、条件筛选及信息整合
5. **展示推理过程**：所有答案均会在最终结果前附上详细的`<think>`推理轨迹

## 数据集格式

每条样本包含以下字段：

| 字段名 | 说明 |
|---|---|
| `instruction` | 针对表格的自然语言问题 |
| `input` | Markdown格式的表格 |
| `response` | 以`<think>...</think>`包裹的推理轨迹，后跟最终答案 |
| `domain` | 表格所属的真实场景领域（例如`healthcare_appointments`（医疗预约）） |
| `question_type` | 12种问题类型之一（详见下文） |
| `n_rows` / `n_cols` | 表格的维度（行数/列数） |
| `numeric_cols` / `categorical_cols` | 列类型元数据 |

### 示例

instruction: "所有门店的总营收是多少？"
input: "| 门店 | 营收 | 员工数 |
|---|---|---|
| 市中心店 | 45000 | 12 |
| 商场店 | 62000 | 18 |
| 机场店 | 38000 | 8 |"
response: "<think>
我需要对营收列求和。

市中心店：45000
商场店：62000
机场店：38000

45000 + 62000 = 107000
107000 + 38000 = 145000
</think>

所有门店的总营收为**145,000美元**。"

## 问题类型

本数据集在12种问题类型中保持均衡分布（训练集每种约3333条，验证集每种约333条）：

| 类型 | 说明 | 答案验证方式 |
|---|---|
| `sum` | 对数值列求和 | 通过Pandas计算验证 |
| `mean` | 数值列的平均值 | 通过Pandas计算验证 |
| `max_row` | 数值最高的行 | 通过Pandas计算验证 |
| `min_row` | 数值最低的行 | 通过Pandas计算验证 |
| `filtered_sum` | 带筛选条件的求和 | 通过Pandas计算验证 |
| `filtered_count` | 带筛选条件的计数 | 通过Pandas计算验证 |
| `percentage` | 符合条件的行占比 | 通过Pandas计算验证 |
| `rank_top3` | 按数值排序的前3行 | 通过Pandas计算验证 |
| `comparison` | 多行数值比较 | 由大语言模型生成 |
| `lookup` | 查找指定单元格的值 | 由大语言模型生成 |
| `compound` | 结合多种操作的复合问题 | 由大语言模型生成 |
| `summarization` | 对整个表格进行总结 | 由大语言模型生成 |

**计算类题型**（从`sum`到`rank_top3`）的答案均经过数学验证——标准答案通过Pandas计算得出，大语言模型仅需生成推理轨迹以解释计算过程，这确保了数值结果的准确性。

## 推理轨迹质量标准

本数据集中的所有`<think>`块均遵循严格的质量规范：
- **精确引用单元格值**：内容直接从表格中逐字复制，绝不进行释义改写
- **分步算术推导**：计算过程按增量方式展示（`a + b = X; X + c = Y`），绝不直接跳至最终结果
- **显式枚举行项**：计数任务需按名称列出所有相关行，而非直接声称总数量
- **仅统计数据行**：百分比与计数计算会正确排除表头行

生成过程中已过滤掉推理轨迹敷衍、占位或不完整的样本。

## 覆盖领域

35个多样化的真实场景领域确保模型能够泛化适配不同的表格结构与词汇体系：
`医疗预约`（healthcare_appointments）· `社交媒体营销活动`（social_media_campaign）· `员工人事`（employee_hr）· `电商商品`（ecommerce_products）· `学生成绩`（student_grades）· `项目进度跟踪`（project_tracking）· `零售门店业绩`（retail_store_performance）· `金融交易`（financial_transactions）· `运动队统计`（sports_team_stats）· `库存管理`（inventory_management）· `客户支持`（customer_support）· `营销线索`（marketing_leads）· `活动报名`（event_registration）· `餐厅菜单`（restaurant_menu）· `航班运营`（flight_operations）· `气象站数据`（weather_station）· `房产挂牌`（real_estate_listings）· `电影票房`（movie_box_office）· `医院检验结果`（hospital_lab_results）· `能源消耗`（energy_consumption）· `健身房会员`（gym_membership）· `农业产量`（agriculture_yield）· `货运物流`（shipping_logistics）· `音乐流媒体`（music_streaming）· `车辆车队`（vehicle_fleet）· `在线课程`（online_courses）· `呼叫中心`（call_center）· `污染监测`（pollution_monitoring）· `酒店预订`（hotel_bookings）· `制造业质检`（manufacturing_qc）· `图书馆流通`（library_circulation）· `慈善捐赠`（charity_donations）· `应用商店分析`（app_store_analytics）· `公共交通`（public_transit）· `野生动物调查`（wildlife_survey）

## 使用方法

python
from datasets import load_dataset

ds = load_dataset("cetusian/markdown-table-expert")

# 训练划分集
train = ds["train"] # 40,000条样本
val = ds["validation"] # 4,000条样本

# 按问题类型筛选
math_samples = train.filter(lambda x: x["question_type"] in ["sum", "mean", "percentage"])

# 按领域筛选
healthcare_samples = train.filter(lambda x: x["domain"] == "healthcare_appointments")

### 微调适配

本数据集专为大语言模型的监督微调（Supervised Fine-Tuning, SFT）设计。`<think>`标签可直接用于思维链训练，若你的训练框架会自动添加此类标签，也可将其移除。

python
# 适配对话式微调格式
def format_sample(sample):
return {
"messages": [
{"role": "user", "content": f"{sample['instruction']}

{sample['input']}"},
{"role": "assistant", "content": sample["response"]},
]
}

## 统计信息

| 划分集 | 样本数量 | 覆盖领域 | 问题类型数量 |
|---|---|---|
| 训练集 | 40,000 | 35 | 12（均衡分布） |
| 验证集 | 4,000 | 35 | 12（均衡分布） |
| **总计** | **44,000** | **35** | **12** |

## 许可证

Apache 2.0