MicPie/unpredictable_cluster04

---
annotations_creators:
- no-annotation
language_creators:
- found
language:
- en
license:
- apache-2.0
multilinguality:
- monolingual
pretty_name: UnpredicTable-cluster04
size_categories:
- 100K<n<1M
source_datasets: []
task_categories:
- multiple-choice
- question-answering
- zero-shot-classification
- text2text-generation
- table-question-answering
- text-generation
- text-classification
- tabular-classification
task_ids:
- multiple-choice-qa
- extractive-qa
- open-domain-qa
- closed-domain-qa
- closed-book-qa
- open-book-qa
- language-modeling
- multi-class-classification
- natural-language-inference
- topic-classification
- multi-label-classification
- tabular-multi-class-classification
- tabular-multi-label-classification
---

# Dataset Card for "UnpredicTable-cluster04" - Dataset of Few-shot Tasks from Tables

## Table of Contents
- [Dataset Description](#dataset-description)
- [Dataset Summary](#dataset-summary)
- [Supported Tasks](#supported-tasks-and-leaderboards)
- [Languages](#languages)
- [Dataset Structure](#dataset-structure)
- [Data Instances](#data-instances)
- [Data Fields](#data-instances)
- [Data Splits](#data-instances)
- [Dataset Creation](#dataset-creation)
- [Curation Rationale](#curation-rationale)
- [Source Data](#source-data)
- [Annotations](#annotations)
- [Personal and Sensitive Information](#personal-and-sensitive-information)
- [Considerations for Using the Data](#considerations-for-using-the-data)
- [Social Impact of Dataset](#social-impact-of-dataset)
- [Discussion of Biases](#discussion-of-biases)
- [Other Known Limitations](#other-known-limitations)
- [Additional Information](#additional-information)
- [Dataset Curators](#dataset-curators)
- [Licensing Information](#licensing-information)
- [Citation Information](#citation-information)

## Dataset Description

- **Homepage:** https://ethanperez.net/unpredictable
- **Repository:** https://github.com/JunShern/few-shot-adaptation
- **Paper:** Few-shot Adaptation Works with UnpredicTable Data
- **Point of Contact:** [e-mail], [e-mail]

### Dataset Summary

The UnpredicTable dataset consists of web tables formatted as few-shot tasks for fine-tuning language models to improve their few-shot performance.

There are several dataset versions available:

* [UnpredicTable-full](https://huggingface.co/datasets/MicPie/unpredictable_full): Starting from the initial WTC corpus of 50M tables, we apply our tables-to-tasks procedure to produce our resulting dataset, [UnpredicTable-full](https://huggingface.co/datasets/MicPie/unpredictable_full), which comprises 413,299 tasks from 23,744 unique websites.

* [UnpredicTable-unique](https://huggingface.co/datasets/MicPie/unpredictable_unique): This is the same as [UnpredicTable-full](https://huggingface.co/datasets/MicPie/unpredictable_full) but filtered to have a maximum of one task per website. [UnpredicTable-unique](https://huggingface.co/datasets/MicPie/unpredictable_unique) contains exactly 23,744 tasks from 23,744 websites.

* [UnpredicTable-5k](https://huggingface.co/datasets/MicPie/unpredictable_5k): This dataset contains 5k random tables from the full dataset.

* UnpredicTable data subsets based on a manual human quality rating (please see our publication for details of the ratings):
* [UnpredicTable-rated-low](https://huggingface.co/datasets/MicPie/unpredictable_rated-low)
* [UnpredicTable-rated-medium](https://huggingface.co/datasets/MicPie/unpredictable_rated-medium)
* [UnpredicTable-rated-high](https://huggingface.co/datasets/MicPie/unpredictable_rated-high)

* UnpredicTable data subsets based on the website of origin:
* [UnpredicTable-baseball-fantasysports-yahoo-com](https://huggingface.co/datasets/MicPie/unpredictable_baseball-fantasysports-yahoo-com)
* [UnpredicTable-bulbapedia-bulbagarden-net](https://huggingface.co/datasets/MicPie/unpredictable_bulbapedia-bulbagarden-net)
* [UnpredicTable-cappex-com](https://huggingface.co/datasets/MicPie/unpredictable_cappex-com)
* [UnpredicTable-cram-com](https://huggingface.co/datasets/MicPie/unpredictable_cram-com)
* [UnpredicTable-dividend-com](https://huggingface.co/datasets/MicPie/unpredictable_dividend-com)
* [UnpredicTable-dummies-com](https://huggingface.co/datasets/MicPie/unpredictable_dummies-com)
* [UnpredicTable-en-wikipedia-org](https://huggingface.co/datasets/MicPie/unpredictable_en-wikipedia-org)
* [UnpredicTable-ensembl-org](https://huggingface.co/datasets/MicPie/unpredictable_ensembl-org)
* [UnpredicTable-gamefaqs-com](https://huggingface.co/datasets/MicPie/unpredictable_gamefaqs-com)
* [UnpredicTable-mgoblog-com](https://huggingface.co/datasets/MicPie/unpredictable_mgoblog-com)
* [UnpredicTable-mmo-champion-com](https://huggingface.co/datasets/MicPie/unpredictable_mmo-champion-com)
* [UnpredicTable-msdn-microsoft-com](https://huggingface.co/datasets/MicPie/unpredictable_msdn-microsoft-com)
* [UnpredicTable-phonearena-com](https://huggingface.co/datasets/MicPie/unpredictable_phonearena-com)
* [UnpredicTable-sittercity-com](https://huggingface.co/datasets/MicPie/unpredictable_sittercity-com)
* [UnpredicTable-sporcle-com](https://huggingface.co/datasets/MicPie/unpredictable_sporcle-com)
* [UnpredicTable-studystack-com](https://huggingface.co/datasets/MicPie/unpredictable_studystack-com)
* [UnpredicTable-support-google-com](https://huggingface.co/datasets/MicPie/unpredictable_support-google-com)
* [UnpredicTable-w3-org](https://huggingface.co/datasets/MicPie/unpredictable_w3-org)
* [UnpredicTable-wiki-openmoko-org](https://huggingface.co/datasets/MicPie/unpredictable_wiki-openmoko-org)
* [UnpredicTable-wkdu-org](https://huggingface.co/datasets/MicPie/unpredictable_wkdu-org)

* UnpredicTable data subsets based on clustering (for the clustering details please see our publication):
* [UnpredicTable-cluster00](https://huggingface.co/datasets/MicPie/unpredictable_cluster00)
* [UnpredicTable-cluster01](https://huggingface.co/datasets/MicPie/unpredictable_cluster01)
* [UnpredicTable-cluster02](https://huggingface.co/datasets/MicPie/unpredictable_cluster02)
* [UnpredicTable-cluster03](https://huggingface.co/datasets/MicPie/unpredictable_cluster03)
* [UnpredicTable-cluster04](https://huggingface.co/datasets/MicPie/unpredictable_cluster04)
* [UnpredicTable-cluster05](https://huggingface.co/datasets/MicPie/unpredictable_cluster05)
* [UnpredicTable-cluster06](https://huggingface.co/datasets/MicPie/unpredictable_cluster06)
* [UnpredicTable-cluster07](https://huggingface.co/datasets/MicPie/unpredictable_cluster07)
* [UnpredicTable-cluster08](https://huggingface.co/datasets/MicPie/unpredictable_cluster08)
* [UnpredicTable-cluster09](https://huggingface.co/datasets/MicPie/unpredictable_cluster09)
* [UnpredicTable-cluster10](https://huggingface.co/datasets/MicPie/unpredictable_cluster10)
* [UnpredicTable-cluster11](https://huggingface.co/datasets/MicPie/unpredictable_cluster11)
* [UnpredicTable-cluster12](https://huggingface.co/datasets/MicPie/unpredictable_cluster12)
* [UnpredicTable-cluster13](https://huggingface.co/datasets/MicPie/unpredictable_cluster13)
* [UnpredicTable-cluster14](https://huggingface.co/datasets/MicPie/unpredictable_cluster14)
* [UnpredicTable-cluster15](https://huggingface.co/datasets/MicPie/unpredictable_cluster15)
* [UnpredicTable-cluster16](https://huggingface.co/datasets/MicPie/unpredictable_cluster16)
* [UnpredicTable-cluster17](https://huggingface.co/datasets/MicPie/unpredictable_cluster17)
* [UnpredicTable-cluster18](https://huggingface.co/datasets/MicPie/unpredictable_cluster18)
* [UnpredicTable-cluster19](https://huggingface.co/datasets/MicPie/unpredictable_cluster19)
* [UnpredicTable-cluster20](https://huggingface.co/datasets/MicPie/unpredictable_cluster20)
* [UnpredicTable-cluster21](https://huggingface.co/datasets/MicPie/unpredictable_cluster21)
* [UnpredicTable-cluster22](https://huggingface.co/datasets/MicPie/unpredictable_cluster22)
* [UnpredicTable-cluster23](https://huggingface.co/datasets/MicPie/unpredictable_cluster23)
* [UnpredicTable-cluster24](https://huggingface.co/datasets/MicPie/unpredictable_cluster24)
* [UnpredicTable-cluster25](https://huggingface.co/datasets/MicPie/unpredictable_cluster25)
* [UnpredicTable-cluster26](https://huggingface.co/datasets/MicPie/unpredictable_cluster26)
* [UnpredicTable-cluster27](https://huggingface.co/datasets/MicPie/unpredictable_cluster27)
* [UnpredicTable-cluster28](https://huggingface.co/datasets/MicPie/unpredictable_cluster28)
* [UnpredicTable-cluster29](https://huggingface.co/datasets/MicPie/unpredictable_cluster29)
* [UnpredicTable-cluster-noise](https://huggingface.co/datasets/MicPie/unpredictable_cluster-noise)

### Supported Tasks and Leaderboards

Since the tables come from the web, the distribution of tasks and topics is very broad. The shape of our dataset is very wide, i.e., we have 1000's of tasks, while each task has only a few examples, compared to most current NLP datasets which are very deep, i.e., 10s of tasks with many examples. This implies that our dataset covers a broad range of potential tasks, e.g., multiple-choice, question-answering, table-question-answering, text-classification, etc.

The intended use of this dataset is to improve few-shot performance by fine-tuning/pre-training on our dataset.

### Languages

English

## Dataset Structure

### Data Instances

Each task is represented as a jsonline file and consists of several few-shot examples. Each example is a dictionary containing a field 'task', which identifies the task, followed by an 'input', 'options', and 'output' field. The 'input' field contains several column elements of the same row in the table, while the 'output' field is a target which represents an individual column of the same row. Each task contains several such examples which can be concatenated as a few-shot task. In the case of multiple choice classification, the 'options' field contains the possible classes that a model needs to choose from.

There are also additional meta-data fields such as 'pageTitle', 'title', 'outputColName', 'url', 'wdcFile'.

### Data Fields

'task': task identifier

'input': column elements of a specific row in the table.

'options': for multiple choice classification, it provides the options to choose from.

'output': target column element of the same row as input.

'pageTitle': the title of the page containing the table.

'outputColName': output column name

'url': url to the website containing the table

'wdcFile': WDC Web Table Corpus file

### Data Splits

The UnpredicTable datasets do not come with additional data splits.

## Dataset Creation

### Curation Rationale

Few-shot training on multi-task datasets has been demonstrated to improve language models' few-shot learning (FSL) performance on new tasks, but it is unclear which training tasks lead to effective downstream task adaptation. Few-shot learning datasets are typically produced with expensive human curation, limiting the scale and diversity of the training tasks available to study. As an alternative source of few-shot data, we automatically extract 413,299 tasks from diverse internet tables. We provide this as a research resource to investigate the relationship between training data and few-shot learning.

### Source Data

#### Initial Data Collection and Normalization

We use internet tables from the English-language Relational Subset of the WDC Web Table Corpus 2015 (WTC). The WTC dataset tables were extracted from the July 2015 Common Crawl web corpus (http://webdatacommons.org/webtables/2015/EnglishStatistics.html). The dataset contains 50,820,165 tables from 323,160 web domains. We then convert the tables into few-shot learning tasks. Please see our publication for more details on the data collection and conversion pipeline.

#### Who are the source language producers?

The dataset is extracted from [WDC Web Table Corpora](http://webdatacommons.org/webtables/).

### Annotations

#### Annotation process

Manual annotation was only carried out for the [UnpredicTable-rated-low](https://huggingface.co/datasets/MicPie/unpredictable_rated-low),
[UnpredicTable-rated-medium](https://huggingface.co/datasets/MicPie/unpredictable_rated-medium), and [UnpredicTable-rated-high](https://huggingface.co/datasets/MicPie/unpredictable_rated-high) data subsets to rate task quality. Detailed instructions of the annotation instructions can be found in our publication.

#### Who are the annotators?

Annotations were carried out by a lab assistant.

### Personal and Sensitive Information

The data was extracted from [WDC Web Table Corpora](http://webdatacommons.org/webtables/), which in turn extracted tables from the [Common Crawl](https://commoncrawl.org/). We did not filter the data in any way. Thus any user identities or otherwise sensitive information (e.g., data that reveals racial or ethnic origins, sexual orientations, religious beliefs, political opinions or union memberships, or locations; financial or health data; biometric or genetic data; forms of government identification, such as social security numbers; criminal history, etc.) might be contained in our dataset.

## Considerations for Using the Data

### Social Impact of Dataset

This dataset is intended for use as a research resource to investigate the relationship between training data and few-shot learning. As such, it contains high- and low-quality data, as well as diverse content that may be untruthful or inappropriate. Without careful investigation, it should not be used for training models that will be deployed for use in decision-critical or user-facing situations.

### Discussion of Biases

Since our dataset contains tables that are scraped from the web, it will also contain many toxic, racist, sexist, and otherwise harmful biases and texts. We have not run any analysis on the biases prevalent in our datasets. Neither have we explicitly filtered the content. This implies that a model trained on our dataset may potentially reflect harmful biases and toxic text that exist in our dataset.

### Other Known Limitations

No additional known limitations.

## Additional Information

### Dataset Curators
Jun Shern Chan, Michael Pieler, Jonathan Jao, Jérémy Scheurer, Ethan Perez

### Licensing Information
Apache 2.0

### Citation Information

```
@misc{chan2022few,
author = {Chan, Jun Shern and Pieler, Michael and Jao, Jonathan and Scheurer, Jérémy and Perez, Ethan},
title = {Few-shot Adaptation Works with UnpredicTable Data},
publisher={arXiv},
year = {2022},
url = {https://arxiv.org/abs/2208.01009}
}
```

原始信息汇总

数据集概述

数据集名称

• 名称: UnpredicTable-cluster04
• 别名: Dataset of Few-shot Tasks from Tables

数据集属性

• 语言: 英语
• 许可证: Apache-2.0
• 多语言性: 单语种
• 大小: 100K<n<1M

数据集版本

• UnpredicTable-full: 包含413,299任务，来自23,744个独特网站。
• UnpredicTable-unique: 与UnpredicTable-full相同，但每个网站最多一个任务。
• UnpredicTable-5k: 包含5k随机表格。
• UnpredicTable-rated-low/medium/high: 根据人工质量评级划分。
• UnpredicTable-clusterXX: 基于聚类分析的多个子集。

任务类型

• 支持的任务: 多选题、问答、零样本分类、文本生成、表格问答、文本分类、表格分类等。
• 任务ID: 包括多种QA、语言建模、多类分类、自然语言推理、主题分类等。

数据集结构

• 数据实例: 每个任务以jsonline文件形式存在，包含多个few-shot示例。
• 数据字段: 包括任务标识、输入、选项、输出、页面标题、输出列名、URL、WDC文件等。
• 数据分割: 不提供额外数据分割。

数据集创建

• 采集理由: 用于研究训练数据与少样本学习之间的关系。
• 源数据: 来自WDC Web Table Corpus 2015的英语关系子集。
• 注释: 仅对特定子集进行人工注释以评估任务质量。
• 个人信息和敏感信息: 数据未经过滤，可能包含敏感信息。

使用考虑

• 社会影响: 作为研究资源，不适用于决策关键或用户面对的情况。
• 偏见讨论: 数据集可能包含有害偏见和文本，未经分析和过滤。

附加信息

• 数据集管理员: Jun Shern Chan, Michael Pieler, Jonathan Jao, Jérémy Scheurer, Ethan Perez
• 许可证信息: Apache 2.0
• 引用信息: 见提供的引用格式。

搜集汇总

数据集介绍

构建方式

在自然语言处理领域，少样本学习性能的提升高度依赖于训练数据的多样性与规模。UnpredicTable-cluster04数据集源自大规模网络表格语料库WDC Web Table Corpus 2015，该语料库包含超过5000万张表格，覆盖32万余个网络域名。研究团队通过自动化流程将这些网络表格转化为少样本学习任务，具体方法是将表格中的每一行视为一个任务实例，选取若干列作为输入，另一列作为目标输出，从而构建出结构化的少样本任务。本数据集是UnpredicTable系列中基于聚类算法划分的子集之一，旨在聚焦特定主题或格式的任务集群，为探究训练数据与少样本适应性能之间的关系提供精细化研究资源。

特点

该数据集的核心特点在于其任务分布的广泛性与稀疏性，涵盖多项选择、问答、文本分类、表格问答等多种任务类型，且每个任务仅包含少量示例，形成‘宽而浅’的数据形态。作为聚类子集，cluster04汇聚了来自相似网络来源或语义主题的表格任务，这有助于研究者分析特定领域数据对模型少样本适应能力的影响。数据集无需人工标注，完全从互联网表格自动提取，因此规模庞大且成本低廉，但同时也可能包含噪音、偏见或不当内容，要求使用者审慎评估数据质量与潜在风险。

使用方法

本数据集主要用于语言模型的少样本微调或预训练，以提升模型在未见任务上的少样本泛化能力。使用时，每个任务以JSON Lines格式存储，包含任务标识符、输入字段、选项字段（适用于多选分类）、输出字段以及页面标题、URL等元数据。研究者可直接加载这些任务实例，将其拼接为少样本提示序列，用于模型训练或评估。由于数据集未提供预设的训练/测试划分，建议用户根据研究目标自行分割，并注意对数据中的潜在偏见和噪音进行必要过滤与分析。

背景与挑战

背景概述

在自然语言处理领域，少样本学习（Few-shot Learning）一直是提升语言模型泛化能力的关键挑战。传统上，构建高质量的多任务少样本数据集依赖昂贵的人工标注，这限制了数据的规模和多样性。为突破这一瓶颈，纽约大学的研究人员Jun Shern Chan、Michael Pieler等人于2022年提出了UnpredicTable系列数据集，其中cluster04子集源自对WDC Web Table Corpus 2015中互联网表格的自动化处理，将超过50万张网络表格转换为413,299个少样本任务。该数据集覆盖了广泛的主题和任务类型，如问答、文本分类和表格推理，旨在为研究训练数据与少样本适应之间的关系提供大规模、多样化的资源，对推动少样本学习的理论探索和实践应用产生了重要影响。

当前挑战

当前数据集面临多重挑战。首先，在领域问题层面，少样本学习需要模型从极少量示例中快速适应新任务，而UnpredicTable-cluster04中的任务分布极广且每个任务示例数有限，导致模型难以捕捉稳定的模式，容易过拟合或欠拟合。其次，在构建过程中，从网络表格自动转换任务时，数据质量参差不齐，包含大量噪声、不完整或语义模糊的表项，且未经过滤的表格可能携带有害偏见或敏感信息，影响模型的公平性和可靠性。此外，缺乏人工标注验证使得任务的有效性难以保证，进一步增加了下游应用的风险和不确定性。

常用场景

经典使用场景

在自然语言处理与少样本学习的研究领域中，MicPie/unpredictable_cluster04 数据集作为 UnpredicTable 系列的一个聚类子集，其经典使用场景在于为语言模型提供丰富且多样化的少样本任务训练素材。该数据集从海量网络表格中自动提取并转化为结构化的少样本任务，涵盖多选题、问答、文本分类、表格问答等多种任务类型。研究者可将其作为微调或预训练的数据源，以增强模型在面对全新任务时的少样本适应能力。其独特之处在于任务数量庞大但每个任务示例稀少，这种“宽而浅”的数据形态有助于探索训练数据多样性与少样本学习泛化能力之间的内在关联。

衍生相关工作

该数据集的发布催生了一系列围绕少样本适应与任务多样性分析的重要研究工作。其基础论文《Few-shot Adaptation Works with UnpredicTable Data》系统论证了大规模网络表格作为少样本训练源的可行性与有效性，并提出了基于聚类、质量评级和来源网站等多维度的数据子集划分方案，为后续研究提供了标准化基准。后续工作进一步探索了基于该数据集的任务选择策略、数据噪声对少样本性能的影响，以及将表格任务与自然语言指令进行对齐的方法。此外，该数据集也被用作评估模型跨任务迁移能力的测试平台，推动了少样本学习领域从单一任务评估向多任务泛化评估的范式转变。

数据集最近研究

最新研究方向

当前，基于大规模网络表格自动生成少样本任务的数据集成为提升语言模型泛化能力的前沿探索方向。UnpredicTable-cluster04作为该系列中按聚类划分的子集，聚焦于从异构互联网表格中提取结构化少样本任务，以研究预训练数据多样性与下游任务适应性的深层关联。这一方向呼应了少样本学习领域对大规模、低成本任务数据的迫切需求，突破了传统人工构建数据集的规模与多样性瓶颈。通过将网络表格转化为涵盖多选问答、文本分类、表格推理等多种任务类型的训练实例，该数据集为探索任务分布、聚类特性与模型零样本迁移能力之间的相互作用提供了宝贵资源，其影响力体现在推动更高效、更通用的少样本适应方法的发展，并引发对数据质量、偏差及社会影响的深入讨论。