Simulation-Based Evaluation of the Performances of an Algorithm for Detecting Abnormal Disease-Related Features in Cattle Mortality Records

We performed a simulation study to evaluate the performances of an anomaly detection algorithm considered in the frame of an automated surveillance system of cattle mortality. The method consisted in a combination of temporal regression and spatial cluster detection which allows identifying, for a given week, clusters of spatial units showing an excess of deaths in comparison with their own historical fluctuations. First, we simulated 1,000 outbreaks of a disease causing extra deaths in the French cattle population (about 200,000 herds and 20 million cattle) according to a model mimicking the spreading patterns of an infectious disease and injected these disease-related extra deaths in an authentic mortality dataset, spanning from January 2005 to January 2010. Second, we applied our algorithm on each of the 1,000 semi-synthetic datasets to identify clusters of spatial units showing an excess of deaths considering their own historical fluctuations. Third, we verified if the clusters identified by the algorithm did contain simulated extra deaths in order to evaluate the ability of the algorithm to identify unusual mortality clusters caused by an outbreak. Among the 1,000 simulations, the median duration of simulated outbreaks was 8 weeks, with a median number of 5,627 simulated deaths and 441 infected herds. Within the 12-week trial period, 73% of the simulated outbreaks were detected, with a median timeliness of 1 week, and a mean of 1.4 weeks. The proportion of outbreak weeks flagged by an alarm was 61% (i.e. sensitivity) whereas one in three alarms was a true alarm (i.e. positive predictive value). The performances of the detection algorithm were evaluated for alternative combination of epidemiologic parameters. The results of our study confirmed that in certain conditions automated algorithms could help identifying abnormal cattle mortality increases possibly related to unidentified health events.

本研究开展模拟实验，以评估纳入牛只死亡自动化监测系统框架的异常检测算法（anomaly detection algorithm）性能。该算法结合时间回归与空间聚类检测技术，可针对特定周度识别出死亡数相较自身历史波动水平偏高的空间单元集群。首先，我们基于模拟传染病传播模式的模型，在法国牛群（约20万牧场、2000万头牛）中模拟了1000次会引发额外死亡的疫病暴发场景，并将这些与疫病相关的超额死亡数据嵌入至时间跨度为2005年1月至2010年1月的真实牛只死亡数据集当中。其次，我们将所提算法应用于这1000个半合成数据集（semi-synthetic datasets）的每一个，以识别死亡数相较自身历史波动水平偏高的空间单元集群。第三，我们验证算法识别出的集群是否包含模拟产生的超额死亡数据，以此评估算法识别由疫病暴发引发的异常死亡集群的能力。在1000次模拟实验中，模拟疫病暴发的持续时间中位数为8周，模拟超额死亡数中位数为5627例，受感染牧场中位数为441个。在12周的试验周期内，73%的模拟疫病暴发被成功检出，检出及时性的中位数为1周，平均及时性为1.4周。被警报标记的暴发周度占比为61%（即灵敏度（sensitivity）），而每3次警报中便有1次为真警报（即阳性预测值（positive predictive value））。本研究还针对不同流行病学参数组合场景，对该检测算法的性能展开评估。本研究结果证实，在特定条件下，自动化算法可助力识别可能与未被发现的健康事件相关的牛只死亡数异常升高情况。