cetusian/markdown-table-qa-01

--- dataset_info: features: - name: id dtype: string - name: instruction dtype: string - name: input dtype: string - name: response dtype: string - name: domain dtype: string - name: question_type dtype: string - name: n_rows dtype: int64 - name: n_cols dtype: int64 - name: numeric_cols list: string - name: categorical_cols list: string splits: - name: train num_examples: 2000 - name: validation num_examples: 200 configs: - config_name: default data_files: - split: train path: data/train-* - split: validation path: data/validation-* --- # Markdown Table QA Dataset — Part 01/20 Part **1** of a 20-dataset collection for training and evaluating language models on structured table understanding and computational reasoning. Each part contains **2,200 samples** (2,000 train + 200 validation) with step-by-step reasoning traces. See the full collection: [cetusian/markdown-table-qa-01](https://huggingface.co/datasets/cetusian/markdown-table-qa-01) through [cetusian/markdown-table-qa-20](https://huggingface.co/datasets/cetusian/markdown-table-qa-20) Parent dataset: [cetusian/markdown-table-qa](https://huggingface.co/datasets/cetusian/markdown-table-qa) (11,000 samples) --- ## What's in it Each sample contains a markdown table paired with a natural language question and a detailed answer with step-by-step reasoning: | Field | Description | |---|---| | `instruction` | Natural language question about the table | | `input` | The markdown table | | `response` | Answer with `<think>...</think>` reasoning trace followed by a final answer | | `domain` | Table domain (e.g. `healthcare_appointments`, `wildlife_survey`) | | `question_type` | One of 12 types — equally balanced (~167 train + ~17 val per type) | ### Reasoning format Every response includes a detailed `<think>` block that: - Quotes **exact cell values** from the table - Shows **all arithmetic step by step** (`a + b = c; c + d = e`) - Enumerates rows explicitly by name for counting tasks - Never skips to final results --- ## Question types (equally balanced) | Type | Description | |---|---| | `sum` | Sum a numeric column | | `mean` | Average of a numeric column | | `max_row` | Row with highest value | | `min_row` | Row with lowest value | | `filtered_sum` | Sum with a filter condition | | `filtered_count` | Count with a filter condition | | `percentage` | Percentage of rows matching a condition | | `rank_top3` | Top 3 rows by a numeric column | | `comparison` | Compare values between two rows | | `lookup` | Look up a specific cell value | | `compound` | Multi-part question combining lookups | | `summarization` | Summarize the entire table | Computational types have **mathematically verified answers** computed with pandas. --- ## Domains 35 real-world domains covering diverse table structures including healthcare, finance, sports, e-commerce, energy, wildlife, logistics, and more. --- ## How to use ```python from datasets import load_dataset ds = load_dataset("cetusian/markdown-table-qa-01") # Load all 20 parts from datasets import concatenate_datasets all_train = concatenate_datasets([ load_dataset(f"cetusian/markdown-table-qa-{i:02d}", split="train") for i in range(1, 21) ]) # -> 40,000 training samples ``` --- ## Generation Generated using a pipeline built on **[vLLM](https://github.com/vllm-project/vllm)** with **OpenAI gpt-oss-120b** (4 GPUs, tensor parallelism). Quality-filtered for proper reasoning traces, answer grounding, and balanced type distribution. --- ## About Surogate **[Surogate](https://surogate.ai)** is a full-stack AgentOps platform for developing, deploying, evaluating, and monitoring reliable AI agents — built by [Invergent AI](https://github.com/invergent-ai/surogate).数据集信息： 特征字段： - 字段名：id，数据类型：字符串（string） - 字段名：instruction，数据类型：字符串（string） - 字段名：input，数据类型：字符串（string） - 字段名：response，数据类型：字符串（string） - 字段名：domain，数据类型：字符串（string） - 字段名：question_type，数据类型：字符串（string） - 字段名：n_rows，数据类型：64位整数（int64） - 字段名：n_cols，数据类型：64位整数（int64） - 字段名：numeric_cols，数据类型：字符串列表（list[string]） - 字段名：categorical_cols，数据类型：字符串列表（list[string]） 数据集划分： - 划分名称：train，样本数量：2000 - 划分名称：validation，样本数量：200 配置项： - 配置名：default，数据文件路径： - 训练划分：data/train-* - 验证划分：data/validation-* # Markdown表格问答数据集 (Markdown Table QA Dataset) — 第01/20部分 本数据集为20个数据集合集的第1部分，用于训练与评估语言模型的结构化表格理解与计算推理能力。每个子数据集包含2200条样本（2000条训练样本 + 200条验证样本），且均附带逐步推理轨迹。 完整合集可通过以下链接访问：[cetusian/markdown-table-qa-01](https://huggingface.co/datasets/cetusian/markdown-table-qa-01) 至 [cetusian/markdown-table-qa-20](https://huggingface.co/datasets/cetusian/markdown-table-qa-20)。 父数据集：[cetusian/markdown-table-qa](https://huggingface.co/datasets/cetusian/markdown-table-qa)（共11000条样本） --- ## 数据集内容说明 每条样本均包含一个Markdown表格、一段自然语言问题，以及一份附带逐步推理轨迹的详细答案： | 字段名 | 描述 | |---|---| | `instruction` | 针对该表格的自然语言问题 | | `input` | Markdown表格本体 | | `response` | 包含`<think>...</think>`推理轨迹与最终答案的详细回答 | | `domain` | 表格所属领域（例如`healthcare_appointments`（医疗预约场景）、`wildlife_survey`（野生动物调查场景）） | | `question_type` | 属于12种题型之一，各类题型样本分布均衡（训练集每种题型约167条，验证集约17条） | ### 推理轨迹格式规范 所有`response`字段均包含完整的`<think>`推理块，该块需满足： - 引用表格中的**精确单元格值** - 完整展示**所有算术运算步骤**（格式如`a + b = c; c + d = e`） - 针对计数任务显式枚举所有目标行的名称 - 绝不跳过至最终结果，需完整呈现推导过程 --- ## 题型说明（样本分布均衡） | 题型 | 描述 | |---|---| | `sum` | 对指定数值列求和 | | `mean` | 计算指定数值列的平均值 | | `max_row` | 找出取值最高的行 | | `min_row` | 找出取值最低的行 | | `filtered_sum` | 带筛选条件的求和计算 | | `filtered_count` | 带筛选条件的计数计算 | | `percentage` | 计算符合指定条件的行占总样本的百分比 | | `rank_top3` | 按指定数值列排序，取出前3行 | | `comparison` | 对比两行的指定数值 | | `lookup` | 查询指定单元格的数值 | | `compound` | 结合多种查询逻辑的复合问题 | | `summarization` | 对整个表格内容进行总结 | 所有计算类题型的答案均通过pandas完成数学验证。 --- ## 覆盖领域 本数据集涵盖35个真实世界领域，包含多样化的表格结构，涉及医疗、金融、体育、电商、能源、野生动物、物流等多个场景。 --- ## 使用方法 python from datasets import load_dataset ds = load_dataset("cetusian/markdown-table-qa-01") # 加载全部20个子数据集 from datasets import concatenate_datasets all_train = concatenate_datasets([ load_dataset(f"cetusian/markdown-table-qa-{i:02d}", split="train") for i in range(1, 21) ]) # 最终将获得40000条训练样本 --- ## 数据集生成说明 本数据集基于**vLLM (vLLM)**与OpenAI gpt-oss-120b构建的流水线生成（使用4张GPU，开启张量并行），并经过质量过滤，以确保推理轨迹规范、答案有据可依且题型分布均衡。 --- ## 关于Surogate **Surogate**是由[Invergent AI](https://github.com/invergent-ai/surogate)开发的全栈AI智能体 (AI Agent) 运维（AgentOps）平台，用于开发、部署、评估与监控可靠的AI智能体。