Pre-Processed Power Grid Frequency Time Series

<strong>Overview</strong><br> This repository contains ready-to-use frequency time series as well as the corresponding pre-processing scripts in python. The data covers three synchronous areas of the European power grid: Continental Europe Great Britain Nordic This work is part of the paper "Predictability of Power Grid Frequency"[1]. Please cite this paper, when using the data and the code. For a detailed documentation of the pre-processing procedure we refer to the supplementary material of the paper. <strong>Data sources</strong><br> We downloaded the frequency recordings from publically available repositories of three different Transmission System Operators (TSOs). <strong>Continental Europe </strong>[2]: We downloaded the data from the German TSO <em>TransnetBW GmbH,</em> which retains the Copyright on the data, but allows to re-publish it upon request [3]. <strong>Great Britain </strong>[4]: The download was supported by National Grid ESO Open Data, which belongs to the British TSO <em>National Grid</em>. They publish the frequency recordings under the NGESO Open License [5]. <strong>Nordic</strong> [6]: We obtained the data from the Finish TSO <em>Fingrid</em>, which provides the data under the open license CC-BY 4.0 [7]. <strong>Content of the repository</strong> <strong>A) Scripts</strong> In the "Download_scripts" folder you will find three scripts to automatically download frequency data from the TSO's websites. In "convert_data_format.py" we save the data with corrected timestamp formats. Missing data is marked as NaN (processing step (1) in the supplementary material of [1]). In "clean_corrupted_data.py" we load the converted data and identify corrupted recordings. We mark them as NaN and clean some of the resulting data holes (processing step (2) in the supplementary material of [1]). The python scripts run with Python 3.7 and with the packages found in "requirements.txt". <strong>B) Yearly converted and cleansed data</strong><br> The folders "&lt;year&gt;_converted" contain the output of "convert_data_format.py" and "&lt;year&gt;_cleansed" contain the output of "clean_corrupted_data.py". <strong>File type</strong>: The files are zipped csv-files, where each file comprises one year. <strong>Data format</strong>: The files contain two columns. The second column contains the frequency values in Hz. The first one represents the time stamps in the format <em>Year-Month-Day Hour-Minute-Second</em>, which is given as naive local time. The local time refers to the following time zones and includes Daylight Saving Times (python time zone in brackets): TransnetBW: Continental European Time (<em>CE)</em> Nationalgrid: Great Britain (<em>GB</em>) Fingrid: Finland (<em>Europe/Helsinki</em>) <strong>NaN representation</strong>: We mark corrupted and missing data as "NaN" in the csv-files. <strong>Use cases</strong><br> We point out that this repository can be used in two different was: <strong>Use pre-processed data</strong>: You can directly use the converted or the cleansed data. Note however, that both data sets include segments of NaN-values due to missing and corrupted recordings. Only a very small part of the NaN-values were eliminated in the cleansed data to not manipulate the data too much. <strong>Produce your own cleansed data</strong>: Depending on your application, you might want to cleanse the data in a custom way. You can easily add your custom cleansing procedure in "clean_corrupted_data.py" and then produce cleansed data from the raw data in "&lt;year&gt;_converted". <strong>License</strong> This work is licensed under multiple licenses, which are located in the "LICENSES" folder. We release the code in the folder "Scripts" under the MIT license . The pre-processed data in the subfolders "**/Fingrid" and "**/Nationalgrid" are licensed under CC-BY 4.0. TransnetBW originally did not publish their data under an open license. We have explicitly received the permission to publish the pre-processed version from TransnetBW. However, we cannot publish our pre-processed version under an open license due to the missing license of the original TransnetBW data. <strong>Changelog</strong><br> Version 2: Add time zone information to description Include new frequency data Update references Change folder structure to yearly folders