Two Daily Weather Datasets: Chiang Mai International Airport and Theodore Francis Green State Airport
收藏资源简介：
Two daily weather datasets for experimenting data-driven models on two different weather types: 1. Chiang Mai International Airport, Chiang Mai, Thailand from January 1st 1998 to July 31st 2019. The data were acquired from the station via personal communication. The following files are provided: - chiang_mai_1998-2019_raw.csv : the raw data. - chiang_mai_1998-2019.csv : the preprocessed data: the dates and redundant variables were removed, the missing data were imputed with MICE algorithm and all units were changed to SI units. 2. Theodore Francis Green State Airport, Providence, RI from January 1st 2006 to October 31st 2019. The data were acquired from the National Oceanic and Atmospheric Administration (https://www.ncdc.noaa.gov/cdo-web/datatools/lcd). The following files are provided: - providence_2006-2019_raw.csv : the raw data. - providence_2006-2019.csv : the preprocessed data: the dates were removed, the missing data were imputed with MICE algorithm and all units were changed to SI units. Additionally, we provide code in Python and shell scripts for reproducibility of the three autoencoder models in "Short-term Daily Precipitation Forecasting with Seasonally-Integrated Autoencoder". The code have the following requirements: - Python 3.6 or higher - Keras 2.2 or higher (Python library) - Tensorflow 1.x.y or where x.y is 12.0 or higher (Python library) The proposed model can be trained by simply running the following command: ./Providence.sh After the training is done, the RMSE and CORR scores will be reported, and the forecast values will be saved in `path/to/data_XXXXXX-xxxxxx.csv`. The README.md file provides additional information on code usage. #### Changing arguments You can modify the arguments in the script files. For example, `--model` and `horizon` let you specify the model and the forecast horizon, respectively. The descriptions of all available options can be accessed via the command: python3 main.py -h #### Running the script in different modes We have prepared the scripts for `prediction` mode and `evaluation` mode, namely `Providence_predict.sh` and `Providence_eval.sh`, as well as the pretrained weights for all three models in the `model` folder. To use these two modes, you need to specify the location of the pretrained weights using `--load` option. For example, the weights of SSAE that makes forecast over the next three days on Providence dataset are stored in `pvd_ssae_3.h5` --load model/pvd_ssae_3.h5 You also need to specify the test data. --test_data Data/data_name.csv REFERENCES: - Liu, L., Jiang, H., He, P., Chen, W., Liu, X., Gao, J., Han, J., 2019. On the variance of the adaptive learning rate and beyond. arXiv preprint arXiv:1908.03265. - Zaytar, M.A., Amrani, C.E., 2016. Sequence to sequence weather forecasting with long short-term memory recurrent neural networks. International Journal of Computer Applications 143, 7–11. doi:10.5120/ijca2016910497.
本数据集包含两份每日天气数据集，用于针对两类不同天气场景开展数据驱动模型的实验： 1. 泰国清迈府清迈国际机场数据集，采集时段为1998年1月1日至2019年7月31日。该数据通过私人通讯渠道从对应气象站获取。本次提供以下文件： - chiang_mai_1998-2019_raw.csv：原始数据文件。 - chiang_mai_1998-2019.csv：预处理后的数据文件，已移除日期字段与冗余变量，采用多重插补链式方程（Multiple Imputation by Chained Equations, MICE）算法补全缺失值，并将所有单位统一转换为国际单位制（SI units）。 2. 美国罗德岛州普罗维登斯市西奥多·弗朗西斯·格林州立机场数据集，采集时段为2006年1月1日至2019年10月31日。该数据源自美国国家海洋和大气管理局（National Oceanic and Atmospheric Administration, NOAA，https://www.ncdc.noaa.gov/cdo-web/datatools/lcd）。本次提供以下文件： - providence_2006-2019_raw.csv：原始数据文件。 - providence_2006-2019.csv：预处理后的数据文件，已移除日期字段，采用MICE算法补全缺失值，并将所有单位统一转换为国际单位制。 此外，本数据集附带用于复现《结合季节集成自编码器的短期每日降水预报》一文中三款自编码器模型的Python代码与Shell脚本。代码运行依赖以下环境： - Python 3.6及以上版本 - Keras 2.2及以上版本（Python库） - TensorFlow 1.x.y（其中x.y≥12.0，Python库） 仅需执行如下命令即可完成模型训练：./Providence.sh 训练完成后，程序将输出均方根误差（Root Mean Square Error, RMSE）与相关系数（Correlation, CORR）评分，并将预报结果保存至`path/to/data_XXXXXX-xxxxxx.csv`。README.md文件提供了代码使用的额外说明。 #### 参数修改 您可修改脚本文件中的参数。例如，`--model`与`horizon`参数可分别指定待使用的模型与预报时效。所有可用参数的说明可通过以下命令查看： python3 main.py -h #### 不同运行模式 我们已分别为「预测」模式与「评估」模式准备了对应脚本`Providence_predict.sh`与`Providence_eval.sh`，并在`model`文件夹中存储了三款模型的预训练权重。若需使用这两种模式，需通过`--load`参数指定预训练权重的路径。例如，针对普罗维登斯数据集训练的、可实现未来3天预报的季节集成自编码器（Seasonally-Integrated Autoencoder, SSAE），其权重存储于`pvd_ssae_3.h5`，可通过`--load model/pvd_ssae_3.h5`加载。同时需通过`--test_data`参数指定测试数据路径：`--test_data Data/data_name.csv` 参考文献： - Liu, L., Jiang, H., He, P., Chen, W., Liu, X., Gao, J., Han, J., 2019. On the variance of the adaptive learning rate and beyond. arXiv preprint arXiv:1908.03265. - Zaytar, M.A., Amrani, C.E., 2016. Sequence to sequence weather forecasting with long short-term memory recurrent neural networks. International Journal of Computer Applications 143, 7–11. doi:10.5120/ijca2016910497.
