DataSheet1_Vaccine Safety Surveillance Using Routinely Collected Healthcare Data—An Empirical Evaluation of Epidemiological Designs.PDF

Background: Routinely collected healthcare data such as administrative claims and electronic health records (EHR) can complement clinical trials and spontaneous reports to detect previously unknown risks of vaccines, but uncertainty remains about the behavior of alternative epidemiologic designs to detect and declare a true risk early.
Methods: Using three claims and one EHR database, we evaluate several variants of the case-control, comparative cohort, historical comparator, and self-controlled designs against historical vaccinations using real negative control outcomes (outcomes with no evidence to suggest that they could be caused by the vaccines) and simulated positive control outcomes.
Results: Most methods show large type 1 error, often identifying false positive signals. The cohort method appears either positively or negatively biased, depending on the choice of comparator index date. Empirical calibration using effect-size estimates for negative control outcomes can bring type 1 error closer to nominal, often at the cost of increasing type 2 error. After calibration, the self-controlled case series (SCCS) design most rapidly detects small true effect sizes, while the historical comparator performs well for strong effects.
Conclusion: When applying any method for vaccine safety surveillance we recommend considering the potential for systematic error, especially due to confounding, which for many designs appears to be substantial. Adjusting for age and sex alone is likely not sufficient to address differences between vaccinated and unvaccinated, and for the cohort method the choice of index date is important for the comparability of the groups. Analysis of negative control outcomes allows both quantification of the systematic error and, if desired, subsequent empirical calibration to restore type 1 error to its nominal value. In order to detect weaker signals, one may have to accept a higher type 1 error.

研究背景：常规收集的医疗健康数据，如行政理赔数据与电子健康记录（electronic health records, EHR），可辅助临床试验与自发报告系统，用于识别此前未被发现的疫苗相关风险，但学界对于早期检测并确认真实风险的替代流行病学研究设计的表现仍存在不确定性。研究方法：本研究依托3个理赔数据库与1个电子健康记录数据库，基于真实阴性对照结局（无证据表明可由疫苗引发的结局）与模拟阳性对照结局，针对既往疫苗接种情况，对病例对照、比较队列、历史对照以及自身对照等多种设计变体展开评估。研究结果：多数方法表现出较高的I类错误（type 1 error）率，常识别出假阳性信号。队列研究方法的偏倚方向取决于对照索引日期的选择，可呈现正向或负向偏倚。基于阴性对照结局的效应量估计进行经验校准，可使I类错误率趋近预设的名义水准，但通常会以增加II类错误（type 2 error）率为代价。经校准后，自身对照病例系列（self-controlled case series, SCCS）设计可最快速地检测到微小的真实效应量，而历史对照设计则对较强效应的检测表现优异。研究结论：在应用任何疫苗安全监测方法时，我们建议需考虑系统误差的潜在影响，尤其是由混杂偏倚引发的系统误差，该偏倚在诸多研究设计中表现显著。仅针对年龄与性别进行调整，可能不足以平衡接种疫苗与未接种疫苗人群间的差异；对于队列研究方法而言，索引日期的选择对组间可比性至关重要。通过对阴性对照结局的分析，既可量化系统误差，亦可在有需求时通过后续经验校准将I类错误率恢复至名义水准。若要检测较弱的信号，则可能需要接受更高的I类错误率。