MicPie/unpredictable_wkdu-org

---
annotations_creators:
- no-annotation
language_creators:
- found
language:
- en
license:
- apache-2.0
multilinguality:
- monolingual
pretty_name: UnpredicTable-wkdu-org
size_categories:
- 100K<n<1M
source_datasets: []
task_categories:
- multiple-choice
- question-answering
- zero-shot-classification
- text2text-generation
- table-question-answering
- text-generation
- text-classification
- tabular-classification
task_ids:
- multiple-choice-qa
- extractive-qa
- open-domain-qa
- closed-domain-qa
- closed-book-qa
- open-book-qa
- language-modeling
- multi-class-classification
- natural-language-inference
- topic-classification
- multi-label-classification
- tabular-multi-class-classification
- tabular-multi-label-classification
---

# Dataset Card for "UnpredicTable-wkdu-org" - Dataset of Few-shot Tasks from Tables

## Table of Contents
- [Dataset Description](#dataset-description)
- [Dataset Summary](#dataset-summary)
- [Supported Tasks](#supported-tasks-and-leaderboards)
- [Languages](#languages)
- [Dataset Structure](#dataset-structure)
- [Data Instances](#data-instances)
- [Data Fields](#data-instances)
- [Data Splits](#data-instances)
- [Dataset Creation](#dataset-creation)
- [Curation Rationale](#curation-rationale)
- [Source Data](#source-data)
- [Annotations](#annotations)
- [Personal and Sensitive Information](#personal-and-sensitive-information)
- [Considerations for Using the Data](#considerations-for-using-the-data)
- [Social Impact of Dataset](#social-impact-of-dataset)
- [Discussion of Biases](#discussion-of-biases)
- [Other Known Limitations](#other-known-limitations)
- [Additional Information](#additional-information)
- [Dataset Curators](#dataset-curators)
- [Licensing Information](#licensing-information)
- [Citation Information](#citation-information)

## Dataset Description

- **Homepage:** https://ethanperez.net/unpredictable
- **Repository:** https://github.com/JunShern/few-shot-adaptation
- **Paper:** Few-shot Adaptation Works with UnpredicTable Data
- **Point of Contact:** [e-mail], [e-mail]

### Dataset Summary

The UnpredicTable dataset consists of web tables formatted as few-shot tasks for fine-tuning language models to improve their few-shot performance.

There are several dataset versions available:

* [UnpredicTable-full](https://huggingface.co/datasets/MicPie/unpredictable_full): Starting from the initial WTC corpus of 50M tables, we apply our tables-to-tasks procedure to produce our resulting dataset, [UnpredicTable-full](https://huggingface.co/datasets/MicPie/unpredictable_full), which comprises 413,299 tasks from 23,744 unique websites.

* [UnpredicTable-unique](https://huggingface.co/datasets/MicPie/unpredictable_unique): This is the same as [UnpredicTable-full](https://huggingface.co/datasets/MicPie/unpredictable_full) but filtered to have a maximum of one task per website. [UnpredicTable-unique](https://huggingface.co/datasets/MicPie/unpredictable_unique) contains exactly 23,744 tasks from 23,744 websites.

* [UnpredicTable-5k](https://huggingface.co/datasets/MicPie/unpredictable_5k): This dataset contains 5k random tables from the full dataset.

* UnpredicTable data subsets based on a manual human quality rating (please see our publication for details of the ratings):
* [UnpredicTable-rated-low](https://huggingface.co/datasets/MicPie/unpredictable_rated-low)
* [UnpredicTable-rated-medium](https://huggingface.co/datasets/MicPie/unpredictable_rated-medium)
* [UnpredicTable-rated-high](https://huggingface.co/datasets/MicPie/unpredictable_rated-high)

* UnpredicTable data subsets based on the website of origin:
* [UnpredicTable-baseball-fantasysports-yahoo-com](https://huggingface.co/datasets/MicPie/unpredictable_baseball-fantasysports-yahoo-com)
* [UnpredicTable-bulbapedia-bulbagarden-net](https://huggingface.co/datasets/MicPie/unpredictable_bulbapedia-bulbagarden-net)
* [UnpredicTable-cappex-com](https://huggingface.co/datasets/MicPie/unpredictable_cappex-com)
* [UnpredicTable-cram-com](https://huggingface.co/datasets/MicPie/unpredictable_cram-com)
* [UnpredicTable-dividend-com](https://huggingface.co/datasets/MicPie/unpredictable_dividend-com)
* [UnpredicTable-dummies-com](https://huggingface.co/datasets/MicPie/unpredictable_dummies-com)
* [UnpredicTable-en-wikipedia-org](https://huggingface.co/datasets/MicPie/unpredictable_en-wikipedia-org)
* [UnpredicTable-ensembl-org](https://huggingface.co/datasets/MicPie/unpredictable_ensembl-org)
* [UnpredicTable-gamefaqs-com](https://huggingface.co/datasets/MicPie/unpredictable_gamefaqs-com)
* [UnpredicTable-mgoblog-com](https://huggingface.co/datasets/MicPie/unpredictable_mgoblog-com)
* [UnpredicTable-mmo-champion-com](https://huggingface.co/datasets/MicPie/unpredictable_mmo-champion-com)
* [UnpredicTable-msdn-microsoft-com](https://huggingface.co/datasets/MicPie/unpredictable_msdn-microsoft-com)
* [UnpredicTable-phonearena-com](https://huggingface.co/datasets/MicPie/unpredictable_phonearena-com)
* [UnpredicTable-sittercity-com](https://huggingface.co/datasets/MicPie/unpredictable_sittercity-com)
* [UnpredicTable-sporcle-com](https://huggingface.co/datasets/MicPie/unpredictable_sporcle-com)
* [UnpredicTable-studystack-com](https://huggingface.co/datasets/MicPie/unpredictable_studystack-com)
* [UnpredicTable-support-google-com](https://huggingface.co/datasets/MicPie/unpredictable_support-google-com)
* [UnpredicTable-w3-org](https://huggingface.co/datasets/MicPie/unpredictable_w3-org)
* [UnpredicTable-wiki-openmoko-org](https://huggingface.co/datasets/MicPie/unpredictable_wiki-openmoko-org)
* [UnpredicTable-wkdu-org](https://huggingface.co/datasets/MicPie/unpredictable_wkdu-org)

* UnpredicTable data subsets based on clustering (for the clustering details please see our publication):
* [UnpredicTable-cluster00](https://huggingface.co/datasets/MicPie/unpredictable_cluster00)
* [UnpredicTable-cluster01](https://huggingface.co/datasets/MicPie/unpredictable_cluster01)
* [UnpredicTable-cluster02](https://huggingface.co/datasets/MicPie/unpredictable_cluster02)
* [UnpredicTable-cluster03](https://huggingface.co/datasets/MicPie/unpredictable_cluster03)
* [UnpredicTable-cluster04](https://huggingface.co/datasets/MicPie/unpredictable_cluster04)
* [UnpredicTable-cluster05](https://huggingface.co/datasets/MicPie/unpredictable_cluster05)
* [UnpredicTable-cluster06](https://huggingface.co/datasets/MicPie/unpredictable_cluster06)
* [UnpredicTable-cluster07](https://huggingface.co/datasets/MicPie/unpredictable_cluster07)
* [UnpredicTable-cluster08](https://huggingface.co/datasets/MicPie/unpredictable_cluster08)
* [UnpredicTable-cluster09](https://huggingface.co/datasets/MicPie/unpredictable_cluster09)
* [UnpredicTable-cluster10](https://huggingface.co/datasets/MicPie/unpredictable_cluster10)
* [UnpredicTable-cluster11](https://huggingface.co/datasets/MicPie/unpredictable_cluster11)
* [UnpredicTable-cluster12](https://huggingface.co/datasets/MicPie/unpredictable_cluster12)
* [UnpredicTable-cluster13](https://huggingface.co/datasets/MicPie/unpredictable_cluster13)
* [UnpredicTable-cluster14](https://huggingface.co/datasets/MicPie/unpredictable_cluster14)
* [UnpredicTable-cluster15](https://huggingface.co/datasets/MicPie/unpredictable_cluster15)
* [UnpredicTable-cluster16](https://huggingface.co/datasets/MicPie/unpredictable_cluster16)
* [UnpredicTable-cluster17](https://huggingface.co/datasets/MicPie/unpredictable_cluster17)
* [UnpredicTable-cluster18](https://huggingface.co/datasets/MicPie/unpredictable_cluster18)
* [UnpredicTable-cluster19](https://huggingface.co/datasets/MicPie/unpredictable_cluster19)
* [UnpredicTable-cluster20](https://huggingface.co/datasets/MicPie/unpredictable_cluster20)
* [UnpredicTable-cluster21](https://huggingface.co/datasets/MicPie/unpredictable_cluster21)
* [UnpredicTable-cluster22](https://huggingface.co/datasets/MicPie/unpredictable_cluster22)
* [UnpredicTable-cluster23](https://huggingface.co/datasets/MicPie/unpredictable_cluster23)
* [UnpredicTable-cluster24](https://huggingface.co/datasets/MicPie/unpredictable_cluster24)
* [UnpredicTable-cluster25](https://huggingface.co/datasets/MicPie/unpredictable_cluster25)
* [UnpredicTable-cluster26](https://huggingface.co/datasets/MicPie/unpredictable_cluster26)
* [UnpredicTable-cluster27](https://huggingface.co/datasets/MicPie/unpredictable_cluster27)
* [UnpredicTable-cluster28](https://huggingface.co/datasets/MicPie/unpredictable_cluster28)
* [UnpredicTable-cluster29](https://huggingface.co/datasets/MicPie/unpredictable_cluster29)
* [UnpredicTable-cluster-noise](https://huggingface.co/datasets/MicPie/unpredictable_cluster-noise)

### Supported Tasks and Leaderboards

Since the tables come from the web, the distribution of tasks and topics is very broad. The shape of our dataset is very wide, i.e., we have 1000's of tasks, while each task has only a few examples, compared to most current NLP datasets which are very deep, i.e., 10s of tasks with many examples. This implies that our dataset covers a broad range of potential tasks, e.g., multiple-choice, question-answering, table-question-answering, text-classification, etc.

The intended use of this dataset is to improve few-shot performance by fine-tuning/pre-training on our dataset.

### Languages

English

## Dataset Structure

### Data Instances

Each task is represented as a jsonline file and consists of several few-shot examples. Each example is a dictionary containing a field 'task', which identifies the task, followed by an 'input', 'options', and 'output' field. The 'input' field contains several column elements of the same row in the table, while the 'output' field is a target which represents an individual column of the same row. Each task contains several such examples which can be concatenated as a few-shot task. In the case of multiple choice classification, the 'options' field contains the possible classes that a model needs to choose from.

There are also additional meta-data fields such as 'pageTitle', 'title', 'outputColName', 'url', 'wdcFile'.

### Data Fields

'task': task identifier

'input': column elements of a specific row in the table.

'options': for multiple choice classification, it provides the options to choose from.

'output': target column element of the same row as input.

'pageTitle': the title of the page containing the table.

'outputColName': output column name

'url': url to the website containing the table

'wdcFile': WDC Web Table Corpus file

### Data Splits

The UnpredicTable datasets do not come with additional data splits.

## Dataset Creation

### Curation Rationale

Few-shot training on multi-task datasets has been demonstrated to improve language models' few-shot learning (FSL) performance on new tasks, but it is unclear which training tasks lead to effective downstream task adaptation. Few-shot learning datasets are typically produced with expensive human curation, limiting the scale and diversity of the training tasks available to study. As an alternative source of few-shot data, we automatically extract 413,299 tasks from diverse internet tables. We provide this as a research resource to investigate the relationship between training data and few-shot learning.

### Source Data

#### Initial Data Collection and Normalization

We use internet tables from the English-language Relational Subset of the WDC Web Table Corpus 2015 (WTC). The WTC dataset tables were extracted from the July 2015 Common Crawl web corpus (http://webdatacommons.org/webtables/2015/EnglishStatistics.html). The dataset contains 50,820,165 tables from 323,160 web domains. We then convert the tables into few-shot learning tasks. Please see our publication for more details on the data collection and conversion pipeline.

#### Who are the source language producers?

The dataset is extracted from [WDC Web Table Corpora](http://webdatacommons.org/webtables/).

### Annotations

#### Annotation process

Manual annotation was only carried out for the [UnpredicTable-rated-low](https://huggingface.co/datasets/MicPie/unpredictable_rated-low),
[UnpredicTable-rated-medium](https://huggingface.co/datasets/MicPie/unpredictable_rated-medium), and [UnpredicTable-rated-high](https://huggingface.co/datasets/MicPie/unpredictable_rated-high) data subsets to rate task quality. Detailed instructions of the annotation instructions can be found in our publication.

#### Who are the annotators?

Annotations were carried out by a lab assistant.

### Personal and Sensitive Information

The data was extracted from [WDC Web Table Corpora](http://webdatacommons.org/webtables/), which in turn extracted tables from the [Common Crawl](https://commoncrawl.org/). We did not filter the data in any way. Thus any user identities or otherwise sensitive information (e.g., data that reveals racial or ethnic origins, sexual orientations, religious beliefs, political opinions or union memberships, or locations; financial or health data; biometric or genetic data; forms of government identification, such as social security numbers; criminal history, etc.) might be contained in our dataset.

## Considerations for Using the Data

### Social Impact of Dataset

This dataset is intended for use as a research resource to investigate the relationship between training data and few-shot learning. As such, it contains high- and low-quality data, as well as diverse content that may be untruthful or inappropriate. Without careful investigation, it should not be used for training models that will be deployed for use in decision-critical or user-facing situations.

### Discussion of Biases

Since our dataset contains tables that are scraped from the web, it will also contain many toxic, racist, sexist, and otherwise harmful biases and texts. We have not run any analysis on the biases prevalent in our datasets. Neither have we explicitly filtered the content. This implies that a model trained on our dataset may potentially reflect harmful biases and toxic text that exist in our dataset.

### Other Known Limitations

No additional known limitations.

## Additional Information

### Dataset Curators
Jun Shern Chan, Michael Pieler, Jonathan Jao, Jérémy Scheurer, Ethan Perez

### Licensing Information
Apache 2.0

### Citation Information

```
@misc{chan2022few,
author = {Chan, Jun Shern and Pieler, Michael and Jao, Jonathan and Scheurer, Jérémy and Perez, Ethan},
title = {Few-shot Adaptation Works with UnpredicTable Data},
publisher={arXiv},
year = {2022},
url = {https://arxiv.org/abs/2208.01009}
}
```

原始信息汇总

数据集概述

数据集名称

• 名称: UnpredicTable-wkdu-org
• 别名: Dataset of Few-shot Tasks from Tables

数据集基本信息

• 语言: 英语
• 许可证: Apache-2.0
• 多语言性: 单语种
• 大小: 100K<n<1M

数据集任务类型

• 任务类型:
  • 多项选择
  • 问答
  • 零样本分类
  • 文本到文本生成
  • 表格问答
  • 文本生成
  • 文本分类
  • 表格分类
• 具体任务ID:
  • 多项选择-问答
  • 抽取式问答
  • 开放域问答
  • 封闭域问答
  • 封闭书问答
  • 开放书问答
  • 语言建模
  • 多类分类
  • 自然语言推理
  • 主题分类
  • 多标签分类
  • 表格多类分类
  • 表格多标签分类

数据集结构

• 数据实例: 每个任务以jsonline文件形式表示，包含多个few-shot示例。每个示例包含task, input, options, output等字段。
• 数据字段:
  • task: 任务标识符
  • input: 表中特定行的列元素
  • options: 多选项分类的选择项
  • output: 与输入同行的目标列元素
  • pageTitle: 包含表格的页面标题
  • outputColName: 输出列名
  • url: 包含表格的网站URL
  • wdcFile: WDC Web Table Corpus文件
• 数据分割: 数据集未提供额外的数据分割。

数据集创建

• 来源数据: 数据集从WDC Web Table Corpus 2015中提取，该数据集包含50,820,165个表格，来自323,160个网站域名。
• 注释过程: 仅对特定子集进行了手动注释以评估任务质量。
• 个人和敏感信息: 数据集可能包含未过滤的个人和敏感信息，因为数据直接从网络表格中提取。

使用数据集的考虑

• 社会影响: 数据集用于研究训练数据与少样本学习之间的关系，可能包含高质量和低质量数据，以及可能不真实或不适当的内容。
• 偏见讨论: 数据集可能包含网络表格中的有害偏见和文本，未进行偏见分析或内容过滤。

附加信息

• 数据集创建者: Jun Shern Chan, Michael Pieler, Jonathan Jao, Jérémy Scheurer, Ethan Perez
• 许可证信息: Apache 2.0
• 引用信息: 请参考论文 "Few-shot Adaptation Works with UnpredicTable Data" 获取详细引用信息。

搜集汇总

数据集介绍

构建方式

在自然语言处理领域，少样本学习任务的构建往往依赖昂贵的人工标注，限制了数据的规模与多样性。为突破这一瓶颈，研究者从WDC Web Table Corpus 2015的英文关系子集中提取了海量互联网表格，通过自动化流程将其转换为少样本学习任务。具体而言，每个表格的行被转化为一个任务实例，其中输入包含同一行的多个列元素，输出则指向另一列的目标值，对于多选分类任务还附加了选项字段。该数据集源自50,820,165个表格，覆盖323,160个网络域名，最终生成413,299个任务，构成了UnpredicTable系列的核心资源。

特点

UnpredicTable-wkdu-org数据集作为UnpredicTable系列的一个子集，其显著特点在于任务来源的广泛性与异构性。由于表格取自互联网，任务类型极为多样，涵盖多项选择、问答、文本分类、表格问答等类别，且每个任务仅包含少量样本，形成了宽度大而深度浅的数据分布。这种结构使得数据集能够为语言模型提供丰富的少样本适应场景，同时保留了来自不同网站的原始元数据，如页面标题、URL和输出列名，便于研究者追溯任务来源与分析数据质量。

使用方法

该数据集旨在通过微调或预训练提升语言模型的少样本性能，使用时可直接加载JSON Lines格式的文件，其中每个任务包含多个示例，每个示例由task、input、options和output字段构成。研究者可将同一任务的多个示例拼接为少样本提示，用于模型训练或评估。数据集不预设训练/测试划分，用户可根据需求自行拆分。此外，HuggingFace上提供了基于质量评级、网站来源或聚类结果的多种子集，便于进行对照实验与消融分析，以探究训练数据对少样本学习效果的影响。

背景与挑战

背景概述

UnpredicTable-wkdu-org数据集由纽约大学的研究人员Jun Shern Chan、Michael Pieler、Jonathan Jao、Jérémy Scheurer和Ethan Perez于2022年创建，旨在探索如何从互联网表格中自动提取少量样本学习任务，以提升语言模型的少样本适应能力。该数据集源于WDC Web Table Corpus 2015，通过将海量网络表格转化为结构化的少样本任务，覆盖了多选问答、文本分类、自然语言推理等广泛的任务类型。其核心研究问题在于揭示训练数据的多样性、质量与下游少样本性能之间的关联，为少样本学习领域提供了大规模、低成本的数据资源，推动了多任务预训练范式的进一步发展。

当前挑战

该数据集面临的挑战主要源于两个方面。在领域问题层面，少样本学习本身受限于任务多样性与数据稀缺性，如何从网络表格中自动构建高质量、覆盖广泛语义的少样本任务，并确保其能有效提升模型泛化能力，是核心难点。在构建过程中，挑战尤为突出：首先，从Common Crawl提取的原始表格噪声极大，包含大量格式混乱、信息残缺或不相关的表格，需设计复杂的过滤与转换流水线；其次，自动生成的任务缺乏人工标注的精确性，可能导致任务定义模糊或输出不一致；此外，网络数据中隐含的偏见、敏感信息及有害内容未被充分过滤，可能对模型训练产生负面影响，增加了数据治理与伦理考量的复杂性。

常用场景

经典使用场景

UnpredicTable-wkdu-org 数据集的核心用途在于为语言模型的少样本学习（Few-shot Learning）提供大规模、多样化的训练任务。该数据集从互联网表格中自动提取，将结构化的表格数据转化为问答、多项选择、文本分类等多种形式的少样本任务，从而有效增强模型在未见任务上的泛化能力。研究者可利用该数据集对预训练语言模型进行微调，显著提升其在少样本场景下的表现，尤其适用于需要快速适应新任务的自然语言处理研究。

解决学术问题

该数据集主要解决了少样本学习中训练数据稀缺且人工标注成本高昂的学术难题。通过自动从海量网络表格中提取任务，UnpredicTable-wkdu-org 突破了传统少样本数据集规模有限、领域单一的局限，为探究训练数据特性与下游任务适应性能之间的关联提供了宝贵资源。其研究意义在于揭示了多样化、低质量数据亦能有效促进模型少样本学习能力，为构建更鲁棒、更通用的语言模型奠定了数据基础。

衍生相关工作

UnpredicTable-wkdu-org 数据集衍生了一系列经典研究工作，包括基于其构建的少样本适应框架（Few-shot Adaptation Works with UnpredicTable Data），该框架系统验证了网络表格数据作为少样本训练资源的有效性。此外，研究者还基于该数据集开发了任务质量评级子集（如 rated-high/medium/low），用于分析训练数据质量对模型性能的影响。这些工作推动了少样本学习领域对数据多样性、噪声容忍度及任务分布的理解，并启发了后续从非结构化文本中自动构建少样本任务的研究方向。