Neighborhood- and Object-Based Probabilistic Verification of the OU MAP Ensemble Forecasts during 2017 and 2018 Hazardous Weather Testbeds Weather and Forecasting

An object-based probabilistic (OBPROB) forecasting framework is developed and applied, together with a more traditional neighborhood-based framework, to convection-permitting ensemble forecasts produced by the University of Oklahoma (OU) Multiscale data Assimilation and Predictability (MAP) laboratory during the 2017 and 2018 NOAA Hazardous Weather Testbed Spring Forecasting Experiments. Case studies from 2017 are used for parameter tuning and demonstration of methodology, while the 2018 ensemble forecasts are systematically verified. The 2017 case study demonstrates that the OBPROB forecast product can provide a unique tool to operational forecasters that includes convective-scale details such as storm mode and morphology, which are typically lost in neighborhood-based methods, while also providing quantitative ensemble probabilistic guidance about those details in a more easily interpretable format than the more commonly used paintball plots. The case study also demonstrates that objective verification metrics reveal different relative performance of the ensemble at different forecast lead times depending on the verification framework (i.e., object versus neighborhood) because of the different features emphasized by object- and neighborhood-based evaluations. Both frameworks are then used for a systematic evaluation of 26 forecasts from the spring of 2018. The OBPROB forecast verification as configured in this study shows less sensitivity to forecast lead time than the neighborhood forecasts. Both frameworks indicate a need for probabilistic calibration to improve ensemble reliability. However, lower ensemble discrimination for OBPROB than the neighborhood-based forecasts is also noted. 2020 Grant no. NA17OAR4590187 Grant no. NA15OAR4590193 Grant no. NA16OAR4590236 CIMMS (Cooperative Institute for Mesoscale Meteorological Studies) OAR (Oceanic and Atmospheric Research) NSSL (National Severe Storms Laboratory) NWS (National Weather Service) SPC (Storm Prediction Center) Submitted https://doi.org/10.1175/WAF-D-19-0060.1 Other 1948

本研究开发并应用了基于对象的概率（object-based probabilistic, OBPROB）预报框架，结合更为传统的基于邻域的预报框架，用于俄克拉荷马大学（University of Oklahoma, OU）多尺度数据同化与可预报性（Multiscale Data Assimilation and Predictability, MAP）实验室在2017年与2018年美国国家海洋和大气管理局（NOAA）危险天气试验台春季预报试验中生成的对流许可集合预报。2017年的案例研究用于参数调优与方法演示，2018年的集合预报则开展系统性验证。2017年的案例研究表明，OBPROB预报产品可为业务预报员提供包含对流尺度细节（如风暴类型与形态）的独特工具——这类细节通常在基于邻域的方法中会丢失——同时相较于更常用的彩弹图（paintball plots），能以更易于解读的形式，为这些细节提供定量化的集合概率指导。该案例研究还证实，由于基于对象与基于邻域的评估所侧重的特征存在差异，客观验证指标会在不同预报提前时效下，展现出集合预报不同的相对性能，具体取决于所采用的验证框架（即对象框架与邻域框架）。随后两种框架均被用于对2018年春季的26次预报开展系统性评估。本研究中配置的OBPROB预报验证相较于邻域预报验证，对预报提前时效的敏感性更低。两种框架均表明，需通过概率校准来提升集合预报的可靠性。但同时也发现，OBPROB的集合判别能力弱于基于邻域的预报。本研究受2020年以下三项资助项目支持：资助号NA17OAR4590187、NA15OAR4590193、NA16OAR4590236。合作单位包括中尺度气象合作研究所（Cooperative Institute for Mesoscale Meteorological Studies, CIMMS）、海洋与大气研究局（Oceanic and Atmospheric Research, OAR）、国家强风暴实验室（National Severe Storms Laboratory, NSSL）、国家气象局（National Weather Service, NWS）与风暴预报中心（Storm Prediction Center, SPC）。本文已提交，DOI：10.1175/WAF-D-19-0060.1，其他编号1948。