TOTRCD
收藏资源简介:
TOTRCD(Temporally-Ordered Tabular Regression Benchmark Suite with Concept Drift)是一个专为表格回归任务设计的基准测试数据集套件,其核心特点是所有数据都包含时间顺序(通过时间列体现)并呈现出概念漂移现象。该套件旨在为研究时间相关的回归模型以及概念漂移的检测与适应提供标准化的评估资源。数据内容方面,TOTRCD整合了来自真实世界的多个公开表格数据集,涵盖空气质量、航班、能源预测、交通流量、房价、天气、医疗监测等多个领域。所有数据集均经过统一格式化处理,字段名称使用标准化前缀进行标识:`Meta::`表示元数据(如标识符、时间戳等,不作为模型输入);`Covariate::Static::`表示静态协变量(时间不变的特征);`Covariate::Temporal::`表示从时间信息派生出的周期性编码特征;`Target::`表示需要预测的目标变量。对于分类变量,采用独热编码,并在列名中使用`::dummy::`中缀进行标记。数据规模上,该套件包含19个子数据集,观测数量从数千到数亿不等(例如,Airlines数据集约有2.26亿条观测,而Coffee Distribution数据集约有6000条观测),特征数量在8到128个之间(经过独热编码后少于1000个)。部分数据集包含多个预测任务(即多个目标变量)。套件中的每个数据集都经过了概念漂移检测流程的验证,确认存在漂移现象。该数据集适用于表格回归任务的研究与评估,特别适合用于探索和测试模型在存在时间依赖性和概念漂移的真实场景下的性能。其严格的纳入标准(包括必须是真实数据、包含时间列、观测数大于5000、特征数有限且存在已验证的概念漂移)保证了数据集的实用性和针对性。
TOTRCD (Temporally-Ordered Tabular Regression Benchmark Suite with Concept Drift) is a benchmark dataset suite specifically designed for tabular regression tasks. Its core characteristic is that all datasets contain temporal order (reflected by the time column) and exhibit concept drift. This suite aims to provide standardized evaluation resources for researching time-related regression models as well as the detection and adaptation of concept drift. In terms of data content, TOTRCD integrates multiple real-world public tabular datasets, covering domains such as air quality, flight, energy forecasting, traffic flow, housing prices, weather, and medical monitoring. All datasets have undergone unified formatting processing, with field names identified using standardized prefixes: `Meta::` denotes metadata (such as identifiers, timestamps, etc., which are not used as model inputs); `Covariate::Static::` denotes static covariates (time-invariant features); `Covariate::Temporal::` denotes periodic encoded features derived from temporal information; `Target::` denotes the target variable to be predicted. For categorical variables, one-hot encoding is adopted, and the `::dummy::` infix is used for marking in column names. In terms of data scale, this suite includes 19 subdatasets, with the number of observations ranging from thousands to hundreds of millions. For example, the Airlines dataset contains approximately 226 million observations, while the Coffee Distribution dataset has approximately 6,000 observations. The number of features ranges from 8 to 128 (fewer than 1,000 after one-hot encoding). Some datasets contain multiple prediction tasks (i.e., multiple target variables). Each dataset in the suite has been verified through a concept drift detection workflow, confirming the presence of concept drift. This dataset is applicable to the research and evaluation of tabular regression tasks, and is particularly suitable for exploring and testing the performance of models in real-world scenarios with temporal dependencies and concept drift. Its strict inclusion criteria—including being real-world data, containing a time column, having more than 5,000 observations, limited feature count, and verified concept drift—ensure the practicality and targeted utility of the dataset.
TOTRCD: 时序表格回归基准套件(含概念漂移)
数据集概述
TOTRCD 是一个包含时间顺序(通过时间列表示)且表现出某种形式概念漂移的表格回归数据集集合。
数据字段结构
数据集列名遵循统一的命名格式,使用前缀标记不同类型的数据:
Meta:::标记标识符、时间戳或排序键,不作为模型输入。例如:Meta::DateTimeCovariate::Static:::标记时间不变的特征输入。例如:Covariate::Static::DistanceCovariate::Temporal:::标记从日期时间得到的周期性编码特征。例如:Covariate::Temporal::Hour (sin)Target:::标记待预测的目标变量。例如:Target::traffic_volume::dummy:::插入在分类变量原名和类别值之间,标记独热编码列。例如:Covariate::Static::weather::dummy::Rain
数据集规模
| 数据集 | 样本数 | 特征数 | Mahalanobis | ASO | ADWIN |
|---|---|---|---|---|---|
| Air Quality | 8,991 | 7,344 | 7,393 | 7,396 | 12 |
| Airlines | 226,082,661 | 8 | ✅ | ❌ | ✅ |
| Appliances Energy Prediction | 19,735 | 29 | ✅ | ✅ | ✅ |
| Beijing PM2.5 | 41,757 | 16 | ✅ | ✅ | ✅ |
| Bike Sharing (Washington DC) | 17,379 | 13 | ✅ | ✅ | ✅ |
| CMAPSS | 53,759 | 61,249 | 25 | ✅ | ✅ |
| Coffee Distribution | 6,016 | 103 | ✅ | ✅ | ✅ |
| Gas Turbine Emission | 36,733 | 36,733 | 9 | ✅ | ✅ |
| Marine Cargo Vessel Power Consumption | 567,442 | 10 | ✅ | ✅ | ✅ |
| Metro Interstate Traffic Volume | 47,942 | 22 | ✅ | ✅ | ✅ |
| Miami Housing 2016 | 13,932 | 12 | ✅ | ✅ | ✅ |
| NOAA Weather | 19,515 | 12 | ✅ | ✅ | ✅ |
| Parking Birmingham | 35,705 | 35 | ✅ | ✅ | ✅ |
| Parkinsons Telemonitoring | 5,875 | 5,875 | 19 | ✅ | ✅ |
| Seoul Bike Sharing Demand | 8,465 | 18 | ✅ | ✅ | ✅ |
| Shifts Weather | 3,544,637 | 128 | ✅ | ❌ | ✅ |
| Steel Industry Energy Consumption | 35,040 | 15 | ✅ | ✅ | ✅ |
| Temperature Forecast | 7,588 | 7,588 | 46 | ✅ | ✅ |
| Tetouan City Power Consumption | 52,416 | 52,416 | 52,416 | 11 | ✅ |
注: 竖线 | 分隔与同一数据集关联的不同任务(即不同目标标签)。
数据集创建标准
入选该套件的数据集必须满足以下所有条件:
- 表格回归任务
- 真实世界数据集,非合成数据
- 包含一个引入自然顺序的时间列
- 在允许重新分发的许可证下公开可用
- 超过5,000个观测值
- 独热编码后特征少于1,000个
- 存在概念漂移证据,并经漂移检测流程确认
许可信息
各数据集的许可证信息如下:
| 数据集 | 许可证 |
|---|---|
| Air Quality | CC BY 4.0 |
| Airlines | CC0 |
| Appliances Energy Prediction | CC BY 4.0 |
| Beijing PM2.5 | CC BY 4.0 |
| Bike Sharing (Washington DC) | CC BY 4.0 |
| CMAPSS | Public Domain |
| Coffee Distribution | Public Domain |
| Gas Turbine Emission | CC BY 4.0 |
| Marine Cargo Vessel Power Consumption | CC BY-NC-SA 4.0 |
| Metro Interstate Traffic Volume | CC BY 4.0 |
| Miami Housing 2016 | CC BY-NC-SA 4.0 |
| NOAA Weather | CC0 |
| Parking Birmingham | CC BY 4.0 |
| Parkinsons Telemonitoring | CC BY 4.0 |
| Seoul Bike Sharing Demand | CC BY 4.0 |
| Shifts Weather | CC BY-NC-SA 4.0 |
| Steel Industry Energy Consumption | CC BY 4.0 |
| Temperature Forecast | CC BY 4.0 |
| Tetouan City Power Consumption | CC BY 4.0 |
引用信息
各数据集的原始论文引用如下:
Air Quality
@article{Vito2008OnFC, title={On field calibration of an electronic nose for benzene estimation in an urban pollution monitoring scenario}, author={Saverio De Vito and Ettore Massera and Marco Piga and Luca Martinotto and Girolamo Di Francia}, journal={Sensors and Actuators B-chemical}, year={2008}, volume={129}, pages={750-757}, url={https://api.semanticscholar.org/CorpusID:94886265} }
Appliances Energy Prediction
@article{Candanedo2017DataDP, title={Data driven prediction models of energy use of appliances in a low-energy house}, author={Luis M. Ibarra Candanedo and Veronique Feldheim and Dominique Deramaix}, journal={Energy and Buildings}, year={2017}, volume={140}, pages={81-97}, url={https://api.semanticscholar.org/CorpusID:63814994} }
Beijing PM2.5
@article{Liang2015AssessingBP, title={Assessing Beijings PM2.5 pollution: severity, weather impact, APEC and winter heating}, author={Xuan Liang and Tao Zou and Bin Guo and Shuo Li and Haozhe Zhang and Shuyi Zhang and Hui Huang and Song Xi Chen}, journal={Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences}, year={2015}, volume={471}, url={https://api.semanticscholar.org/CorpusID:130615236} }
Bike Sharing (Washington DC)
@article{FanaeeT2013EventLC, title={Event labeling combining ensemble detectors and background knowledge}, author={Hadi Fanaee-T and Jo{~a}o Gama}, journal={Progress in Artificial Intelligence}, year={2013}, volume={2}, pages={113 - 127}, url={https://api.semanticscholar.org/CorpusID:256282956} }
Gas Turbine Emission
@article{Kaya2019PredictingCA, title={Predicting CO and NOx emissions from gas turbines: Novel data and a benchmark PEMS}, author={Kaya, Heysem and T{"u}fekci, P{i}nar and Uzun, Erdin{c{c}}}, journal={Turkish Journal of Electrical Engineering and Computer Sciences}, year={2019}, volume={27}, number={6}, pages={4783--4796}, doi={10.3906/elk-1807-87} }
Marine Cargo Vessel Power Consumption
@dataset{malinin_2022_7684813, title={Shifts Marine Cargo Vessel Power Consumption Prediction Dataset}, author={Malinin, Andrey and Athanasopoulos, Andreas and Barakovic, Muhamed and Bach Cuadra, Meritxell and Gales, Mark and Granziera, Cristina and Graziani, Mara and Kartashev, Nikolay and Kyriakopoulos, Konstantinos and Lu, Po-Jui and Molchanova, Nataliia and Nikitakis, Antonis and Raina, Vatsal and La Rosa, Francesco and Sivena, Eli and Tsarsitalidis, Vasileios and Tsompopoulou, Efi and Volf, Elena}, publisher={Zenodo}, month={sep}, year={2022}, version={2.0}, doi={10.5281/zenodo.7684813}, url={https://doi.org/10.5281/zenodo.7684813} }
Metro Interstate Traffic Volume
@misc{metro_interstate_traffic_volume_492, author = {Hogue, John}, title = {{Metro Interstate Traffic Volume}}, year = {2019}, howpublished = {UCI Machine Learning Repository}, note = {{DOI}: https://doi.org/10.24432/C5X60B} }
Miami Housing 2016
@techreport{Mayer2021StructuredAR, title={Structured Additive Regression and Tree Boosting}, author={Mayer, Michael and Bourassa, Steven C. and Hoesli, Martin and Scognamiglio, Donato}, institution={Swiss Finance Institute}, type={Swiss Finance Institute Research Paper}, number={21-83}, year={2021}, doi={10.2139/ssrn.3924412}, url={https://ssrn.com/abstract=3924412} }
Parking Birmingham
@inproceedings{Stolfi2017PredictingCP, title={Predicting Car Park Occupancy Rates in Smart Cities}, author={Stolfi, Daniel H. and Alba, Enrique and Yao, Xin}, booktitle={Smart Cities: Second International Conference, Smart-CT 2017}, address={M{a}laga, Spain}, pages={107--117}, year={2017}, doi={10.1007/978-3-319-59513-9_11} }
Parkinsons Telemonitoring
@article{Tsanas2009AccurateTO, title={Accurate Telemonitoring of Parkinsons Disease Progression by Noninvasive Speech Tests}, author={Athanasios Tsanas and Max A. Little and Patrick E. McSharry and Lorraine O. Ramig}, journal={IEEE Transactions on Biomedical Engineering}, year={2009}, volume={57}, pages={884-893}, url={https://api.semanticscholar.org/CorpusID:7382779} }
Seoul Bike Sharing Demand
@article{Sathishkumar2020UsingDM, title={Using data mining techniques for bike sharing demand prediction in metropolitan city}, author={Sathishkumar, V E and Park, Jangwoo and Cho, Yongyun}, journal={Computer Communications}, year={2020}, volume={153}, pages={353--366}, doi={10.1016/j.comcom.2020.02.007} }
@article{Sathishkumar2020ARB, title={A rule-based model for Seoul Bike sharing demand prediction using weather data}, author={Sathishkumar, V E and Cho, Yongyun}, journal={European Journal of Remote Sensing}, year={2020}, volume={53}, number={sup1}, pages={166--183}, doi={10.1080/22797254.2020.1725789} }
Shifts Weather
@inproceedings{ malinin2021shifts, title={Shifts: A Dataset of Real Distributional Shift Across Multiple Large-Scale Tasks}, author={Andrey Malinin and Neil Band and Yarin Gal and Mark Gales and Alexander Ganshin and German Chesnokov and Alexey Noskov and Andrey Ploskonosov and Liudmila Prokhorenkova and Ivan Provilkov and Vatsal Raina and Vyas Raina and Denis Roginskiy and Mariya Shmatova and Panagiotis Tigas and Boris Yangel}, booktitle={Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2)}, year={2021}, url={https://openreview.net/forum?id=qM45LHaWM6E} }
Steel Industry Energy Consumption
@article{VE2020EfficientEC, title={Efficient energy consumption prediction model for a data analytic-enabled industry building in a smart city}, author={Sathishkumar V E and Changsun Shin and Yongyun Cho}, journal={Building Research & Information}, year={2020}, volume={49}, pages={127 - 143}, url={https://api.semanticscholar.org/CorpusID:224916577} }
Temperature Forecast
@misc{bias_correction_of_numerical_prediction_model_temperature_forecast_514, title = {{Bias correction of numerical prediction model temperature forecast}}, year = {2020}, howpublished = {UCI Machine Learning Repository}, note = {{DOI}: https://doi.org/10.24432/C59K76} }
Tetouan City Power Consumption
@article{Salam2018ComparisonOM, title={Comparison of Machine Learning Algorithms for the Power Consumption Prediction : - Case Study of Tetouan city –}, author={Abdul Rahim Salam and Abdelaaziz El Hibaoui}, journal={2018 6th International Renewable and Sustainable Energy Conference (IRSEC)}, year={2018}, pages={1-5}, url={https://api.semanticscholar.org/CorpusID:145050098} }




