Pre-Processed Power Grid Frequency Time Series (2020-2023)

Overview This dataset provides pre-processed frequency time series data for 2020-2023, covering three synchronous areas of the European power grid: Continental Europe Nordic This work is part of the paper "Probabilistic and Explainable Machine Learning for Tabular Power Grid Data"[1]. Please cite this paper, when using the data and the code. Relationship to Previous Work This dataset extends the time coverage of the original dataset [2], which covered 2012-2021. For a detailed documentation of the pre-processing procedure we refer to the supplementary material of paper "Predictability of Power Grid Frequency"[3]. The same methodology and preprocessing procedures have been applied to maintain consistency and comparability with the original work. Data Sources Continental Europe [4]: To get the data just replace the year and month in the url with the needed one. In June 2022 the frequency data of Continental Europe was moved to [5]. The download and preprocessing scripts were adapted to the new data sources. To distinguish between both data sources, the data from the SG HoBA data [5] was stored in a separate folder. In order to keep the structure with one file per year, combine_2022_ce_data.ipynb notebook was used to combine the data from the first and second half of 2022 and the data was saved in the 2022_cleansed/TransnetBW/2022.zip folder. Nordic [6] We obtained the data from the Finish TSO Fingrid, which provides the data under the open license CC-BY 4.0 [7]. Content of the repository A) Scripts 1) In the `Download_scripts` folder you will find three scripts to automatically download frequency data from the TSO's websites. 2) In `convert_data_format.py` we save the data with corrected timestamp formats. 3) In `clean_corrupted_data.py` we load the converted data and identify corrupted recordings. We mark them as NaN and clean some of the resulting data holes (processing step (2) in the supplementary material of [3]). The python scripts were adapted to run with Python 3.11 and with the packages found in `requirements.txt`. b) Yearly converted and cleansed data The folder `Data_cleansed` contains the output of `clean_corrupted_data.py`. File type: The files are zipped csv-files, where each file comprises one year. Data format: The files contain two columns. The second column contains the frequency values in Hz. The first one represents the time stamps in the format *Year-Month-Day Hour-Minute-Second*, which is given as naive local time. The local time refers to the following time zones and includes Daylight Saving Times (python time zone in brackets): TransnetBW: Continental European Time (*CET*) Fingrid: Finland (*Europe/Helsinki*) NaN representation: We mark corrupted and missing data as "NaN" in the csv-files.### 数据集概述 本数据集提供了2020-2023年经过预处理的电网频率时间序列数据，覆盖欧洲电网的三个同步区域： - 欧洲大陆电网 - 北欧电网 本研究属于论文《面向表格电网数据的概率可解释机器学习》[1]的一部分。使用本数据集与代码时，请引用该论文。 ### 与已有研究的关联 本数据集扩展了原始数据集[2]的时间覆盖范围，原始数据集的时间跨度为2012-2021年。关于预处理流程的详细文档，请参阅论文《电网频率可预测性》[3]的补充材料。为保持与原始研究的一致性和可比性，本研究采用了相同的方法论与预处理流程。 ### 数据来源 #### 欧洲大陆电网[4] 获取数据时只需将URL中的年份和月份替换为所需值即可。2022年6月，欧洲大陆电网的频率数据迁移至[5]。下载与预处理脚本已适配新的数据源。为区分两类数据源，来自SG HoBA数据集[5]的数据被存储于单独文件夹中。为保持“每年一个文件”的结构，我们使用`combine_2022_ce_data.ipynb`脚本合并2022年上下半年的数据，并将合并后的数据保存至`2022_cleansed/TransnetBW/2022.zip`文件夹。 #### 北欧电网[6] 数据来自芬兰输电系统运营商（Transmission System Operator，TSO）Fingrid，该数据以CC-BY 4.0[7]开放许可协议发布。 ### 仓库内容 #### A) 脚本文件 1. 在`Download_scripts`文件夹中，包含三个可自动从输电系统运营商网站下载频率数据的脚本。 2. `convert_data_format.py`用于修正时间戳格式并保存数据。 3. `clean_corrupted_data.py`用于加载转换后的数据，识别损坏的记录，将其标记为NaN，并填补部分由此产生的数据空白（对应文献[3]补充材料中的处理步骤(2)）。 本套Python脚本适配Python 3.11环境，所需依赖包可在`requirements.txt`中查看。 #### b) 经过转换与清洗的年度数据 `Data_cleansed`文件夹包含`clean_corrupted_data.py`的运行输出。 ### 文件类型 所有文件均为压缩CSV文件，每个文件对应一年的数据。 ### 数据格式 文件包含两列数据：第二列为以赫兹（Hz）为单位的频率值；第一列为时间戳，格式为*年-月-日 时-分-秒*，采用无时区信息的本地时间。该本地时间对应以下时区，并包含夏令时调整（括号内为Python时区标识）： - TransnetBW：欧洲大陆时间（*CET*） - Fingrid：芬兰时区（*Europe/Helsinki*） ### 缺失值与损坏数据表示 在CSV文件中，我们将损坏或缺失的数据标记为“NaN”。