WeatherBench 2

WeatherBench 2是一个用于数据驱动全球天气模型的基准数据集，旨在加速天气建模的进展。该数据集包括一个开源评估框架、公开可用的训练、地面实况和基准数据，以及一个持续更新的网站，提供最新的指标和最先进的模型。数据集支持更高分辨率的数据和评估，并增加了额外的度量标准，用于评估全球、中程（1-14天）天气预报的性能。

WeatherBench 2 is a benchmark dataset for data-driven global weather models, aimed at accelerating progress in weather modeling. This dataset includes an open-source evaluation framework, publicly available training, ground truth and benchmark data, as well as a continuously updated website that provides the latest metrics and state-of-the-art models. The dataset supports higher-resolution data and evaluations, and adds additional metrics for assessing the performance of global and medium-range (1-14 day) weather forecasting.

搜集汇总

数据集介绍

构建方式

WeatherBench 2 is meticulously crafted to serve as a comprehensive benchmark for the evaluation of data-driven global weather models, particularly focusing on medium-range forecasts spanning 1 to 14 days. The dataset is constructed by integrating an open-source evaluation framework, publicly accessible training and ground truth data, and a continuously updated website that provides the latest metrics and state-of-the-art models. The evaluation framework adheres to established practices for assessing weather forecasts at leading operational weather centers, ensuring a robust and standardized comparison platform.

特点

WeatherBench 2 distinguishes itself through its high-resolution data support and the inclusion of additional evaluation metrics, which are pivotal for advancing data-driven weather modeling. The dataset emphasizes probabilistic prediction, recognizing the inherent uncertainty in weather forecasting due to chaotic error growth. This focus on probabilistic metrics, such as the Continuous Ranked Probability Score (CRPS) and spread-skill ratio, ensures that the dataset is well-suited for evaluating both deterministic and probabilistic weather forecasts.

使用方法

WeatherBench 2 is designed to be a versatile tool for researchers and practitioners in the field of weather forecasting. Users can leverage the dataset to train and evaluate their models using the provided ground truth data and evaluation code. The dataset supports a wide range of variables and resolutions, allowing for comprehensive model assessments. Additionally, the dynamic and open-source nature of the framework encourages community contributions, facilitating continuous updates and improvements to the benchmark.

背景与挑战

背景概述

WeatherBench 2, an evolution of the original WeatherBench benchmark, was introduced to accelerate advancements in data-driven weather modeling. Developed by a consortium led by Google Research and Google DeepMind, in collaboration with the European Centre for Medium-Range Weather Forecasts, WeatherBench 2 aims to provide a robust evaluation framework for global, medium-range (1–14 day) weather forecasts. The dataset includes an open-source evaluation framework, publicly accessible training and ground truth data, and a continuously updated website featuring the latest metrics and state-of-the-art models. This initiative underscores the growing significance of machine learning in weather prediction, aiming to bridge the gap between traditional physical models and innovative data-driven approaches.

当前挑战

The primary challenge addressed by WeatherBench 2 is the evaluation of data-driven weather models against traditional physical models, particularly in the context of medium-range forecasts. The dataset confronts several key issues: 1) Ensuring the reliability and accuracy of data-driven models in predicting weather variables over extended periods. 2) Addressing the inherent complexity and high dimensionality of weather data, which poses significant challenges in model training and validation. 3) Balancing the need for probabilistic forecasts to account for weather's chaotic nature with the practical requirements of deterministic predictions. 4) The operational feasibility of initializing data-driven models with real-time data, as opposed to reanalysis datasets like ERA5, which are not available in live forecasting scenarios. These challenges highlight the need for a comprehensive and dynamic benchmarking framework to foster continuous improvement in data-driven weather forecasting.

常用场景

经典使用场景

WeatherBench 2 数据集在气象预报领域中被广泛用于评估和比较全球中长期（1-14天）天气预报模型的性能。其经典使用场景包括对物理模型和数据驱动模型的直接预测能力进行基准测试，特别是在高分辨率数据和复杂气象变量的预测上。通过提供公开的评估框架和基准数据，WeatherBench 2 促进了数据驱动天气建模领域的快速发展。

解决学术问题

WeatherBench 2 数据集解决了在数据驱动天气预报模型中常见的学术研究问题，如模型预测的准确性、不确定性和极端天气事件的预测能力。它通过定义一系列关键评分指标，如均方根误差（RMSE）、异常相关系数（ACC）和连续排名概率评分（CRPS），为模型的性能提供了全面的评估。这些指标基于领先的气象中心的实践，确保了评估的科学性和可靠性。

衍生相关工作

WeatherBench 2 数据集的发布催生了一系列相关的经典工作，特别是在深度学习和图神经网络在天气预报中的应用。例如，GraphCast 和 Pangu-Weather 等模型基于该数据集进行了训练和评估，展示了数据驱动方法在天气预报中的潜力。此外，该数据集还促进了混合机器学习-物理模型（如 NeuralGCM）的发展，这些模型结合了数据驱动和物理约束，以提高预测的准确性和可靠性。