VisMetHack2022: Visualizing winds and surface variables from the ECMWF IFS 1-km nature run

<strong>Overview</strong> This data collection was contributed to the Visualisation Hackathon 2022 (#VisMetHack2022), in conjunction with the Using ECMWF's Forecasts (UEF2022) workshop. The European Center for Medium-Range Weather Forecasts (ECMWF) and the Oak Ridge National Laboratory (ORNL) are pleased to announce access to the data collection from global 1-km nature run (NR) simulations using the Integrated Forecast System (IFS) with explicit convection. We invite you to join us in exploring this precursor to a digital twin of the earth! The NR simulations reveal unprecedented detail of the earth’s atmosphere [1], and the then outgoing Editor-in-Chief of AGU JAMES commended the project as one of “stunning ambitions,” enabled by computational capacity at scale [2]. The project also won the <em>2020 HPCwire Readers Choice Award </em>for Best Use of HPC in Physical Sciences. A set of two NR seasonal simulations have been completed, one corresponding to the northern hemispheric winter months (NDJF) and the other for the North Atlantic tropical cyclone season (ASO). The project used the Summit supercomputer at the Oak Ridge Leadership Computing Facility (OLCF). The simulations were facilitated with an INCITE award from the US Department of Energy Office of Science. For the first seasonal run of four months (NDJF), the hydrostatic IFS model was initialized at 00Z on 1 November 2018. The NR for the TC season (AS) was initialized at 00Z on 1 August 2019. The NR simulations were constrained only by sea surface temperatures (SST) at the lower boundary. The IFS output was saved every 3 hours. After feedback and interest from the scientific community, the simulations were rerun for four specific extreme events, with output every 15 minutes. The special cases include a tropical cycle and three severe storm events over the continental USA. <strong>NR Data for visualizing winds</strong> A small subset from the 1-km IFS NR collection is make available for #VisMetHack22. This subset is extracted from the tropical cyclone area in the North Atlantic from the ASO simulations. The 912 model time steps correspond to 97935 to 111600 in minutes since the NR reference time 2019-08-01 00:00:00. The time increment is 15 minutes, corresponding to the output frequency. The following variables are provided for #VisMetHack2022:<br> Short Name Parameter ID Units Long Name 10u 165 m/s 10 metre U wind component 10v 166 m/s 10 metre V wind component 2t 167 K 2 metre temperature i10fg 228029 m/s Instantaneous 10 metre wind gust msl 151 Pa Mean sea level pressure xtprate 99999 kg m**-2 s**-1 Total instantaneous precipitation rate. Summation of convective and large scale rain and snowfall rates. <strong>Data processing</strong> The native model output was in the form of data objects consisting of GRIB1/2 16-bit AEC compressed messages. The messages were extracted from the FDB database instances into one or more files. The files containing the GRIB messages were interpolated to 0.02 x 0.02 a regular latitude-longitude grid using ECMWF Meteorological Interpolation and Regridding (MIR), and then written out to files as GRIB messages. The MIR output files were extracted to the area of interest (AOI) from global fields, and converted to Netcdf-4 (NC). The metadata in NC4 files were selectively edited or added. Finally, the NC4 files were compressed to reduce volume using the ncks utility from Netcdf Operators (NCO), with lossless L1 compression. The variable ‘xtprate’ was calculated by a summation of instantaneous and large scape rainfall and snowfall rates. <strong>Contact</strong> Valentine Anantharaj &lt;[e-mail]&gt; or &lt;[e-mail]&gt; Samuel Hatfield &lt;[e-mail]&gt; <strong>Citation and references</strong> Please cite the following manuscript as well as the DOI provided by Zenodo: [1] Wedi, N. P., Polichtchouk, I., Dueben, P., Anantharaj, V. G., Bauer, P., Boussetta, S., et al. (2020). A baseline for global weather and climate simulations at 1 km resolution. Journal of Advances in Modeling Earth Systems, 12, e2020MS002192. https://doi.org/10.1029/2020MS002192 [2] Anantharaj, V., Hatfield, S. and Vukovic, Milana (2022). VisMetHack2022: Visualizing winds and surface variables from the ECMWF IFS 1-km nature run. https://doi.org/10.5281/zenodo.6633929 <strong>Acknowledgements</strong> This research used resources of the Oak Ridge Leadership Computing Facility, which is a DOE Office of Science User Facility supported under Contract DE-AC05-00OR22725. ECMWF also benefited from collaborations funded via ESCAPE-2 (No. 800897), MAESTRO (No. 801101), EuroEXA (No. 754337), and ESiWACE-2 (No. 823988) projects funded by the European Union's Horizon 2020 future and emerging technologies and the research and innovation programmes.概述 本数据集提交至2022年可视化黑客松（#VisMetHack2022），并与“使用欧洲中期天气预报中心预报”（UEF2022）工作坊同步推出。欧洲中期天气预报中心（European Center for Medium-Range Weather Forecasts, ECMWF）与橡树岭国家实验室（Oak Ridge National Laboratory, ORNL）欣然宣布，开放基于集成预报系统（Integrated Forecast System, IFS）开展的全球1公里分辨率自然运行（Nature Run, NR）模拟数据集的访问权限，该系统采用显式对流方案。诚邀您一同探索这一地球数字孪生的前驱模型！ 该NR模拟揭示了地球大气前所未有的细节[1]，时任美国地球物理联盟（AGU）主编的JAMES将该项目誉为“兼具宏伟抱负”的典范，其实现依托于大规模计算算力[2]。该项目还荣获《2020 HPCwire读者选择奖》“物理科学领域高性能计算最佳应用”奖项。 项目已完成两组NR季节性模拟，一组对应北半球冬季月份（NDJF，即11月至次年2月），另一组对应北大西洋热带气旋季（ASO，即8月至10月）。本项目使用橡树岭领导力计算设施（Oak Ridge Leadership Computing Facility, OLCF）的Summit超级计算机完成，模拟工作获得了美国能源部科学办公室的INCITE项目资助。 第一组为期四个月的季节性模拟（NDJF）于2018年11月1日00Z启动初始化。针对热带气旋季的NR模拟（ASO）则于2019年8月1日00Z启动初始化。NR模拟仅通过下边界的海表温度（Sea Surface Temperature, SST）进行约束。IFS输出每3小时保存一次。在收到科学界的反馈与关注后，项目组针对4起特定极端事件重新开展了模拟，输出频率调整为每15分钟一次。这些特殊案例包括1起热带气旋以及3起美国大陆境内的强风暴事件。 用于风场可视化的NR数据 本次#VisMetHack22开放的数据集为1公里分辨率IFS NR集合的一个小子集，提取自ASO模拟中北大西洋热带气旋活动区域。该子集包含912个模型时间步，对应自NR参考时间2019-08-01 00:00:00起的第97935分钟至第111600分钟，时间步长为15分钟，与输出频率一致。本次#VisMetHack2022提供以下变量： 短名称 参数ID 单位 长名称 10u 165 m/s 10米U风分量 10v 166 m/s 10米V风分量 2t 167 K 2米气温 i10fg 228029 m/s 瞬时10米阵风风速 msl 151 Pa 平均海平面气压 xtprate 99999 kg·m⁻²·s⁻¹ 总瞬时降水率，为对流性降水与大尺度雨雪降水率的总和。 数据处理 原始模型输出为包含GRIB1/2格式16位AEC压缩报文的数据对象。报文从FDB数据库实例中提取为一个或多个文件。通过欧洲中期天气预报中心气象插值与重网格化工具（ECMWF Meteorological Interpolation and Regridding, MIR），将包含GRIB报文的文件插值至0.02°×0.02°的规则经纬网格，随后以GRIB报文格式输出至文件。将MIR输出的全球场文件裁剪至目标区域（Area of Interest, AOI），并转换为Netcdf-4（NC）格式。随后对NC4文件中的元数据进行选择性编辑与补充。最终，借助Netcdf操作工具集（Netcdf Operators, NCO）中的ncks工具，采用无损L1压缩算法对NC4文件进行压缩以减小体积。变量‘xtprate’通过对瞬时降水率与大尺度雨雪降水率求和计算得到。 联系方式 Valentine Anantharaj &lt;[e-mail]&gt; 或 &lt;[e-mail]&gt; Samuel Hatfield &lt;[e-mail]&gt; 引用与参考文献 请引用以下手稿以及Zenodo平台提供的DOI： [1] Wedi, N. P., Polichtchouk, I., Dueben, P., Anantharaj, V. G., Bauer, P., Boussetta, S., 等. (2020). 1公里分辨率全球天气与气候模拟的基准方案. 《地球系统建模进展期刊》, 12, e2020MS002192. https://doi.org/10.1029/2020MS002192 [2] Anantharaj, V., Hatfield, S. 与 Vukovic, Milana (2022). VisMetHack2022: 基于ECMWF IFS 1公里自然运行模拟的风场与地表变量可视化. https://doi.org/10.5281/zenodo.6633929 致谢 本研究使用了橡树岭领导力计算设施的计算资源，该设施是美国能源部科学办公室用户设施，受合同DE-AC05-00OR22725支持。欧洲中期天气预报中心的合作研究也得到了欧盟地平线2020计划未来与新兴技术及研究创新项目资助的ESCAPE-2（编号800897）、MAESTRO（编号801101）、EuroEXA（编号754337）以及ESiWACE-2（编号823988）项目的支持。