inria-soda/tabular-benchmark
收藏Hugging Face2023-09-04 更新2024-03-04 收录
下载链接：
https://hf-mirror.com/datasets/inria-soda/tabular-benchmark
下载链接
链接失效反馈官方服务：
资源简介：
---
annotations_creators: []
license: []
pretty_name: tabular_benchmark
tags: []
task_categories:
- tabular-classification
- tabular-regression
configs:
- config_name: clf_cat_albert
data_files: clf_cat/albert.csv
- config_name: clf_cat_compas-two-years
data_files: clf_cat/compas-two-years.csv
- config_name: clf_cat_covertype
data_files: clf_cat/covertype.csv
- config_name: clf_cat_default-of-credit-card-clients
data_files: clf_cat/default-of-credit-card-clients.csv
- config_name: clf_cat_electricity
data_files: clf_cat/electricity.csv
- config_name: clf_cat_eye_movements
data_files: clf_cat/eye_movements.csv
- config_name: clf_cat_road-safety
data_files: clf_cat/road-safety.csv
- config_name: clf_num_Bioresponse
data_files: clf_num/Bioresponse.csv
- config_name: clf_num_Diabetes130US
data_files: clf_num/Diabetes130US.csv
- config_name: clf_num_Higgs
data_files: clf_num/Higgs.csv
- config_name: clf_num_MagicTelescope
data_files: clf_num/MagicTelescope.csv
- config_name: clf_num_MiniBooNE
data_files: clf_num/MiniBooNE.csv
- config_name: clf_num_bank-marketing
data_files: clf_num/bank-marketing.csv
- config_name: clf_num_california
data_files: clf_num/california.csv
- config_name: clf_num_covertype
data_files: clf_num/covertype.csv
- config_name: clf_num_credit
data_files: clf_num/credit.csv
- config_name: clf_num_default-of-credit-card-clients
data_files: clf_num/default-of-credit-card-clients.csv
- config_name: clf_num_electricity
data_files: clf_num/electricity.csv
- config_name: clf_num_eye_movements
data_files: clf_num/eye_movements.csv
- config_name: clf_num_heloc
data_files: clf_num/heloc.csv
- config_name: clf_num_house_16H
data_files: clf_num/house_16H.csv
- config_name: clf_num_jannis
data_files: clf_num/jannis.csv
- config_name: clf_num_pol
data_files: clf_num/pol.csv
- config_name: reg_cat_Airlines_DepDelay_1M
data_files: reg_cat/Airlines_DepDelay_1M.csv
- config_name: reg_cat_Allstate_Claims_Severity
data_files: reg_cat/Allstate_Claims_Severity.csv
- config_name: reg_cat_Bike_Sharing_Demand
data_files: reg_cat/Bike_Sharing_Demand.csv
- config_name: reg_cat_Brazilian_houses
data_files: reg_cat/Brazilian_houses.csv
- config_name: reg_cat_Mercedes_Benz_Greener_Manufacturing
data_files: reg_cat/Mercedes_Benz_Greener_Manufacturing.csv
- config_name: reg_cat_SGEMM_GPU_kernel_performance
data_files: reg_cat/SGEMM_GPU_kernel_performance.csv
- config_name: reg_cat_abalone
data_files: reg_cat/abalone.csv
- config_name: reg_cat_analcatdata_supreme
data_files: reg_cat/analcatdata_supreme.csv
- config_name: reg_cat_delays_zurich_transport
data_files: reg_cat/delays_zurich_transport.csv
- config_name: reg_cat_diamonds
data_files: reg_cat/diamonds.csv
- config_name: reg_cat_house_sales
data_files: reg_cat/house_sales.csv
- config_name: reg_cat_medical_charges
data_files: reg_cat/medical_charges.csv
- config_name: reg_cat_nyc-taxi-green-dec-2016
data_files: reg_cat/nyc-taxi-green-dec-2016.csv
- config_name: reg_cat_particulate-matter-ukair-2017
data_files: reg_cat/particulate-matter-ukair-2017.csv
- config_name: reg_cat_seattlecrime6
data_files: reg_cat/seattlecrime6.csv
- config_name: reg_cat_topo_2_1
data_files: reg_cat/topo_2_1.csv
- config_name: reg_cat_visualizing_soil
data_files: reg_cat/visualizing_soil.csv
- config_name: reg_num_Ailerons
data_files: reg_num/Ailerons.csv
- config_name: reg_num_Bike_Sharing_Demand
data_files: reg_num/Bike_Sharing_Demand.csv
- config_name: reg_num_Brazilian_houses
data_files: reg_num/Brazilian_houses.csv
- config_name: reg_num_MiamiHousing2016
data_files: reg_num/MiamiHousing2016.csv
- config_name: reg_num_abalone
data_files: reg_num/abalone.csv
- config_name: reg_num_cpu_act
data_files: reg_num/cpu_act.csv
- config_name: reg_num_delays_zurich_transport
data_files: reg_num/delays_zurich_transport.csv
- config_name: reg_num_diamonds
data_files: reg_num/diamonds.csv
- config_name: reg_num_elevators
data_files: reg_num/elevators.csv
- config_name: reg_num_house_16H
data_files: reg_num/house_16H.csv
- config_name: reg_num_house_sales
data_files: reg_num/house_sales.csv
- config_name: reg_num_houses
data_files: reg_num/houses.csv
- config_name: reg_num_medical_charges
data_files: reg_num/medical_charges.csv
- config_name: reg_num_nyc-taxi-green-dec-2016
data_files: reg_num/nyc-taxi-green-dec-2016.csv
- config_name: reg_num_pol
data_files: reg_num/pol.csv
- config_name: reg_num_sulfur
data_files: reg_num/sulfur.csv
- config_name: reg_num_superconduct
data_files: reg_num/superconduct.csv
- config_name: reg_num_wine_quality
data_files: reg_num/wine_quality.csv
- config_name: reg_num_yprop_4_1
data_files: reg_num/yprop_4_1.csv
---
# Tabular Benchmark
## Dataset Description
This dataset is a curation of various datasets from [openML](https://www.openml.org/) and is curated to benchmark performance of various machine learning algorithms.
- **Repository:** https://github.com/LeoGrin/tabular-benchmark/community
- **Paper:** https://hal.archives-ouvertes.fr/hal-03723551v2/document
### Dataset Summary
Benchmark made of curation of various tabular data learning tasks, including:
- Regression from Numerical and Categorical Features
- Regression from Numerical Features
- Classification from Numerical and Categorical Features
- Classification from Numerical Features
### Supported Tasks and Leaderboards
- `tabular-regression`
- `tabular-classification`
## Dataset Structure
### Data Splits
This dataset consists of four splits (folders) based on tasks and datasets included in tasks.
- reg_num: Task identifier for regression on numerical features.
- reg_cat: Task identifier for regression on numerical and categorical features.
- clf_num: Task identifier for classification on numerical features.
- clf_cat: Task identifier for classification on categorical features.
Depending on the dataset you want to load, you can load the dataset by passing `task_name/dataset_name` to `data_files` argument of `load_dataset` like below:
```python
from datasets import load_dataset
dataset = load_dataset("inria-soda/tabular-benchmark", data_files="reg_cat/house_sales.csv")
```
## Dataset Creation
### Curation Rationale
This dataset is curated to benchmark performance of tree based models against neural networks. The process of picking the datasets for curation is mentioned in the paper as below:
- **Heterogeneous columns**. Columns should correspond to features of different nature. This excludes
images or signal datasets where each column corresponds to the same signal on different sensors.
- **Not high dimensional**. We only keep datasets with a d/n ratio below 1/10.
- **Undocumented datasets** We remove datasets where too little information is available. We did keep
datasets with hidden column names if it was clear that the features were heterogeneous.
- **I.I.D. data**. We remove stream-like datasets or time series.
- **Real-world data**. We remove artificial datasets but keep some simulated datasets. The difference is
subtle, but we try to keep simulated datasets if learning these datasets are of practical importance
(like the Higgs dataset), and not just a toy example to test specific model capabilities.
- **Not too small**. We remove datasets with too few features (< 4) and too few samples (< 3 000). For
benchmarks on numerical features only, we remove categorical features before checking if enough
features and samples are remaining.
- **Not too easy**. We remove datasets which are too easy. Specifically, we remove a dataset if a simple model (max of a single tree and a regression, logistic or OLS)
reaches a score whose relative difference with the score of both a default Resnet (from Gorishniy et al. [2021]) and a default HistGradientBoosting model (from scikit learn)
is below 5%. Other benchmarks use different metrics to remove too easy datasets, like removing datasets perfectly separated by a single decision classifier [Bischl et al., 2021],
but this ignores varying Bayes rate across datasets. As tree ensembles are superior to simple trees and logistic regresison [Fernández-Delgado et al., 2014],
a close score for the simple and powerful models suggests that we are already close to the best achievable score.
- **Not deterministic**. We remove datasets where the target is a deterministic function of the data. This
mostly means removing datasets on games like poker and chess. Indeed, we believe that these
datasets are very different from most real-world tabular datasets, and should be studied separately
### Source Data
**Numerical Classification**
|dataset_name|n_samples|n_features|original_link|new_link|
|---|---|---|---|---|
|electricity|38474.0|7.0|https://www.openml.org/d/151|https://www.openml.org/d/44120|
|covertype|566602.0|10.0|https://www.openml.org/d/293|https://www.openml.org/d/44121|
|pol|10082.0|26.0|https://www.openml.org/d/722|https://www.openml.org/d/44122|
|house_16H|13488.0|16.0|https://www.openml.org/d/821|https://www.openml.org/d/44123|
|MagicTelescope|13376.0|10.0|https://www.openml.org/d/1120|https://www.openml.org/d/44125|
|bank-marketing|10578.0|7.0|https://www.openml.org/d/1461|https://www.openml.org/d/44126|
|Bioresponse|3434.0|419.0|https://www.openml.org/d/4134|https://www.openml.org/d/45019|
|MiniBooNE|72998.0|50.0|https://www.openml.org/d/41150|https://www.openml.org/d/44128|
|default-of-credit-card-clients|13272.0|20.0|https://www.openml.org/d/42477|https://www.openml.org/d/45020|
|Higgs|940160.0|24.0|https://www.openml.org/d/42769|https://www.openml.org/d/44129|
|eye_movements|7608.0|20.0|https://www.openml.org/d/1044|https://www.openml.org/d/44130|
|Diabetes130US|71090.0|7.0|https://www.openml.org/d/4541|https://www.openml.org/d/45022|
|jannis|57580.0|54.0|https://www.openml.org/d/41168|https://www.openml.org/d/45021|
|heloc|10000.0|22.0|"https://www.kaggle.com/datasets/averkiyoliabev/home-equity-line-of-creditheloc?select=heloc_dataset_v1+%281%29.csv"|https://www.openml.org/d/45026|
|credit|16714.0|10.0|"https://www.kaggle.com/c/GiveMeSomeCredit/data?select=cs-training.csv"|https://www.openml.org/d/44089|
|california|20634.0|8.0|"https://www.dcc.fc.up.pt/ltorgo/Regression/cal_housing.html"|https://www.openml.org/d/45028|
**Categorical Classification**
|dataset_name|n_samples|n_features|original_link|new_link|
|---|---|---|---|---|
|electricity|38474.0|8.0|https://www.openml.org/d/151|https://www.openml.org/d/44156|
|eye_movements|7608.0|23.0|https://www.openml.org/d/1044|https://www.openml.org/d/44157|
|covertype|423680.0|54.0|https://www.openml.org/d/1596|https://www.openml.org/d/44159|
|albert|58252.0|31.0|https://www.openml.org/d/41147|https://www.openml.org/d/45035|
|compas-two-years|4966.0|11.0|https://www.openml.org/d/42192|https://www.openml.org/d/45039|
|default-of-credit-card-clients|13272.0|21.0|https://www.openml.org/d/42477|https://www.openml.org/d/45036|
|road-safety|111762.0|32.0|https://www.openml.org/d/42803|https://www.openml.org/d/45038|
**Numerical Regression**
|dataset_name|n_samples|n_features|original_link|new_link|
|---|---|---|---|---|
|cpu_act|8192.0|21.0|https://www.openml.org/d/197|https://www.openml.org/d/44132|
|pol|15000.0|26.0|https://www.openml.org/d/201|https://www.openml.org/d/44133|
|elevators|16599.0|16.0|https://www.openml.org/d/216|https://www.openml.org/d/44134|
|wine_quality|6497.0|11.0|https://www.openml.org/d/287|https://www.openml.org/d/44136|
|Ailerons|13750.0|33.0|https://www.openml.org/d/296|https://www.openml.org/d/44137|
|yprop_4_1|8885.0|42.0|https://www.openml.org/d/416|https://www.openml.org/d/45032|
|houses|20640.0|8.0|https://www.openml.org/d/537|https://www.openml.org/d/44138|
|house_16H|22784.0|16.0|https://www.openml.org/d/574|https://www.openml.org/d/44139|
|delays_zurich_transport|5465575.0|9.0|https://www.openml.org/d/40753|https://www.openml.org/d/45034|
|diamonds|53940.0|6.0|https://www.openml.org/d/42225|https://www.openml.org/d/44140|
|Brazilian_houses|10692.0|8.0|https://www.openml.org/d/42688|https://www.openml.org/d/44141|
|Bike_Sharing_Demand|17379.0|6.0|https://www.openml.org/d/42712|https://www.openml.org/d/44142|
|nyc-taxi-green-dec-2016|581835.0|9.0|https://www.openml.org/d/42729|https://www.openml.org/d/44143|
|house_sales|21613.0|15.0|https://www.openml.org/d/42731|https://www.openml.org/d/44144|
|sulfur|10081.0|6.0|https://www.openml.org/d/23515|https://www.openml.org/d/44145|
|medical_charges|163065.0|5.0|https://www.openml.org/d/42720|https://www.openml.org/d/44146|
|MiamiHousing2016|13932.0|14.0|https://www.openml.org/d/43093|https://www.openml.org/d/44147|
|superconduct|21263.0|79.0|https://www.openml.org/d/43174|https://www.openml.org/d/44148|
**Categorical Regression**
|dataset_name|n_samples|n_features|original_link|new_link|
|---|---|---|---|---|
|topo_2_1|8885.0|255.0|https://www.openml.org/d/422|https://www.openml.org/d/45041|
|analcatdata_supreme|4052.0|7.0|https://www.openml.org/d/504|https://www.openml.org/d/44055|
|visualizing_soil|8641.0|4.0|https://www.openml.org/d/688|https://www.openml.org/d/44056|
|delays_zurich_transport|5465575.0|12.0|https://www.openml.org/d/40753|https://www.openml.org/d/45045|
|diamonds|53940.0|9.0|https://www.openml.org/d/42225|https://www.openml.org/d/44059|
|Allstate_Claims_Severity|188318.0|124.0|https://www.openml.org/d/42571|https://www.openml.org/d/45046|
|Mercedes_Benz_Greener_Manufacturing|4209.0|359.0|https://www.openml.org/d/42570|https://www.openml.org/d/44061|
|Brazilian_houses|10692.0|11.0|https://www.openml.org/d/42688|https://www.openml.org/d/44062|
|Bike_Sharing_Demand|17379.0|11.0|https://www.openml.org/d/42712|https://www.openml.org/d/44063|
|Airlines_DepDelay_1M|1000000.0|5.0|https://www.openml.org/d/42721|https://www.openml.org/d/45047|
|nyc-taxi-green-dec-2016|581835.0|16.0|https://www.openml.org/d/42729|https://www.openml.org/d/44065|
|abalone|4177.0|8.0|https://www.openml.org/d/42726|https://www.openml.org/d/45042|
|house_sales|21613.0|17.0|https://www.openml.org/d/42731|https://www.openml.org/d/44066|
|seattlecrime6|52031.0|4.0|https://www.openml.org/d/42496|https://www.openml.org/d/45043|
|medical_charges|163065.0|5.0|https://www.openml.org/d/42720|https://www.openml.org/d/45048|
|particulate-matter-ukair-2017|394299.0|6.0|https://www.openml.org/d/42207|https://www.openml.org/d/44068|
|SGEMM_GPU_kernel_performance|241600.0|9.0|https://www.openml.org/d/43144|https://www.openml.org/d/44069|
### Dataset Curators
Léo Grinsztajn, Edouard Oyallon, Gaël Varoquaux.
### Licensing Information
[More Information Needed]
### Citation Information
Léo Grinsztajn, Edouard Oyallon, Gaël Varoquaux. Why do tree-based models still outperform deep
learning on typical tabular data?. NeurIPS 2022 Datasets and Benchmarks Track, Nov 2022, New
Orleans, United States. ffhal-03723551v2f
提供机构：
inria-soda原始信息汇总
Tabular Benchmark 数据集概述
数据集描述
名称: Tabular Benchmark
目的: 用于评估不同机器学习算法在各种表格数据学习任务上的性能。
任务类型:
- 分类（分类自数值和分类特征、分类自数值特征）
- 回归（回归自数值和分类特征、回归自数值特征）
数据集结构
数据分割
- clf_num: 数值特征分类任务
- clf_cat: 分类特征分类任务
- reg_num: 数值特征回归任务
- reg_cat: 数值和分类特征回归任务
数据文件
分类任务
- clf_num: 包含Bioresponse, Diabetes130US, Higgs等数据集。
- clf_cat: 包含albert, compas-two-years, covertype等数据集。
回归任务
- reg_num: 包含Ailerons, Bike_Sharing_Demand, Brazilian_houses等数据集。
- reg_cat: 包含Airlines_DepDelay_1M, Allstate_Claims_Severity, Bike_Sharing_Demand等数据集。
数据集创建
数据选择标准
- 特征多样性
- 数据维度适中
- 数据信息充足
- 独立同分布数据
- 实际应用数据
- 数据规模适中
- 数据难度适中
- 非确定性数据
数据来源
数据集来源于openML，经过筛选和整理以适应不同的机器学习任务。
数据集维护者
- Léo Grinsztajn
- Edouard Oyallon
- Gaël Varoquaux
许可信息
- [待补充]
引用信息
- Léo Grinsztajn, Edouard Oyallon, Gaël Varoquaux. Why do tree-based models still outperform deep learning on typical tabular data?. NeurIPS 2022 Datasets and Benchmarks Track, Nov 2022, New Orleans, United States.
搜集汇总
数据集介绍
构建方式
在机器学习领域，针对表格数据的算法评测长期缺乏统一且严谨的基准平台。为此，研究者从OpenML平台精心遴选出涵盖多种实际场景的数据集，构建了Tabular Benchmark。该基准的构建遵循一套严格的质量控制准则：确保特征具有异质性，排除图像或信号类数据；限制特征数与样本数之比低于1/10；剔除信息匮乏或人工合成痕迹过重的数据集；移除时间序列及确定性问题，如棋牌类游戏；同时过滤掉样本量过少（少于3000条）或特征维度过低（少于4维）的数据集。此外，通过简单模型与先进模型的性能对比，剔除了过于简单、已接近性能上界的任务。最终，数据集被划分为数值分类、类别分类、数值回归与类别回归四大任务族，共计超过60个独立的评测任务。
特点
该数据集的核心特点在于其系统性与多样性。它囊括了从金融信贷、医疗诊断到交通延误、房产销售等众多领域的真实世界数据，样本规模从数千到数百万不等，特征类型涵盖纯数值与混合类别变量。这种精心设计的异构性，使得研究者能够在统一的框架下，公平且全面地对比决策树集成模型与深度神经网络在典型表格任务上的表现差异。每个子数据集均保留了原始数据的分布特性与噪声水平，避免了过度清洗带来的信息损失，从而更准确地反映算法在实际应用中的泛化能力与鲁棒性。
使用方法
使用该数据集进行评测十分便捷。研究者可通过HuggingFace的`datasets`库加载所需任务，只需在`load_dataset`函数中指定数据集名称`inria-soda/tabular-benchmark`，并通过`data_files`参数传入具体的任务与数据集路径，例如`data_files='reg_cat/house_sales.csv'`，即可自动获取对应的CSV文件。数据加载后，可直接用于训练与评估。该基准的官方代码仓库提供了详细的复现指南，支持用户灵活调用不同任务族，从而系统性地检验模型在分类与回归、纯数值与混合特征等多维场景下的综合性能，为表格数据算法的研究提供了坚实的评测基础。
背景与挑战
背景概述
在机器学习的广阔版图中，表格数据（tabular data）以其普遍性与复杂性占据着举足轻重的地位。然而，长期以来，针对该领域的算法评测常因数据集规模不一、特征类型混杂以及任务目标纷繁而缺乏统一的标杆。为系统性地探究树模型与深度神经网络在典型表格数据上的性能差异，Léo Grinsztajn、Edouard Oyallon 与 Gaël Varoquaux 于2022年联合创建了 Tabular Benchmark 数据集。该数据集精心遴选自 OpenML 平台，涵盖分类与回归两大核心任务，并细分为纯数值特征与混合类别特征两种场景，共包含数十个具有代表性的子数据集。其诞生不仅为 NeurIPS 2022 的相关研究提供了坚实的实验基础，更在学术界引发了关于表格学习范式的深刻反思，成为评估新型算法不可或缺的基准资源。
当前挑战
Tabular Benchmark 的构建与所涉领域问题均面临多重挑战。从领域问题来看，表格数据普遍存在特征异质性高、样本量差异悬殊以及噪声复杂等特性，使得模型泛化能力的公平评测极为困难。为此，数据集严格剔除了高维、小样本、确定性函数映射及过于简单的任务，以确保基准的区分度与实用性。在构建过程中，研究人员需应对数据来源分散、格式不统一及元信息缺失等难题，通过制定详尽的筛选标准（如 d/n 比率、特征数量下限、易学性阈值）来保证子数据集的质量与多样性。此外，将原始数据统一转化为 CSV 格式并划分出四类任务子集，亦是一项繁琐而严谨的工程，旨在为后续研究者提供开箱即用的标准化评测平台。
常用场景
经典使用场景
在表格数据学习领域，inria-soda/tabular-benchmark 数据集被广泛用作基准评测平台，尤其聚焦于对比树模型与神经网络在典型表格分类与回归任务上的表现。该数据集精心筛选了涵盖数值型与类别型特征的多样化子集，确保任务具有异质性列、非高维、非确定性及真实世界背景等特性，从而为研究者提供一个公平且具挑战性的试验场。经典用法包括评估模型在电力负荷预测、信用风险评估、天体物理事件分类等标准任务上的泛化能力，并通过统一的预处理与划分策略，实现不同算法间的可重复比较。
衍生相关工作
围绕该数据集衍生了一系列具有里程碑意义的研究工作，其中最具代表性的是 Grinsztajn 等人于 NeurIPS 2022 发表的论文《Why do tree-based models still outperform deep learning on typical tabular data?》，该文直接基于此基准揭示了树模型与神经网络性能差异的深层机理。后续工作如 TabNet、NODE 以及 SAINT 等新型深度表格学习模型，均以此数据集作为核心评估工具来验证其设计有效性。此外，该基准还催生了关于特征编码策略、缺失值处理以及数据增强技术的系统性对比研究，推动了整个表格学习领域评价体系的标准化进程。
数据集最近研究
最新研究方向
在表格数据建模领域，树模型与深度神经网络的性能较量持续引发学界热议。INRIA-SODA团队推出的Tabular Benchmark数据集，正是为了系统性地探究这一核心问题而构建的权威基准。该数据集精心筛选了来自OpenML的数十个真实世界任务，涵盖数值与类别特征的分类与回归场景，严格排除了高维、过易或具有确定性映射的数据，从而确保评估的公平性与挑战性。当前，该基准已成为验证新型表格深度学习架构（如FT-Transformer、TabNet及各类预训练模型）能否在典型表格数据上超越梯度提升树的关键试验场。其发布直接关联到NeurIPS 2022上关于“树模型为何仍占优”的经典辩论，推动了后续对特征交互建模、数据正则化及混合模型的大量研究，深刻影响了表格数据机器学习的前沿方向与评估标准。
以上内容由遇见数据集搜集并总结生成
