Evaluation and comparison of statistical methods for early temporal detection of outbreaks: A simulation-based study

The objective of this paper is to evaluate a panel of statistical algorithms for temporal outbreak detection. Based on a large dataset of simulated weekly surveillance time series, we performed a systematic assessment of 21 statistical algorithms, 19 implemented in the R package surveillance and two other methods. We estimated false positive rate (FPR), probability of detection (POD), probability of detection during the first week, sensitivity, specificity, negative and positive predictive values and F1-measure for each detection method. Then, to identify the factors associated with these performance measures, we ran multivariate Poisson regression models adjusted for the characteristics of the simulated time series (trend, seasonality, dispersion, outbreak sizes, etc.). The FPR ranged from 0.7% to 59.9% and the POD from 43.3% to 88.7%. Some methods had a very high specificity, up to 99.4%, but a low sensitivity. Methods with a high sensitivity (up to 79.5%) had a low specificity. All methods had a high negative predictive value, over 94%, while positive predictive values ranged from 6.5% to 68.4%. Multivariate Poisson regression models showed that performance measures were strongly influenced by the characteristics of time series. Past or current outbreak size and duration strongly influenced detection performances.

本研究旨在评估一系列用于时序疫情暴发检测的统计算法。基于大规模模拟周度监测时间序列数据集，我们对21种统计算法开展了系统性评估：其中19种实现于R语言surveillance包中，剩余2种为其他独立方法。我们为每种检测方法估算了假阳性率（false positive rate, FPR）、检测概率（probability of detection, POD）、首周检测概率、灵敏度、特异度、阴性预测值、阳性预测值以及F1值。随后，为识别与上述性能指标相关的影响因素，我们构建了针对模拟时间序列特征（趋势、季节性、离散程度、暴发规模等）进行校正的多元泊松回归模型。研究结果显示，假阳性率介于0.7%至59.9%之间，检测概率则处于43.3%至88.7%区间。部分算法具备极高的特异度，最高可达99.4%，但灵敏度较低；而高灵敏度（最高达79.5%）的算法往往伴随较低的特异度。所有算法的阴性预测值均超过94%，阳性预测值则介于6.5%至68.4%之间。多元泊松回归模型结果表明，性能指标受时间序列特征的影响显著，既往或当前暴发的规模与持续时长对检测性能的影响尤为突出。