Challenges in Coding Adverse Events in Clinical Trials: A Systematic Review

BackgroundMisclassification of adverse events in clinical trials can sometimes have serious consequences. Therefore, each of the many steps involved, from a patient's adverse experience to presentation in tables in publications, should be as standardised as possible, minimising the scope for interpretation. Adverse events are categorised by a predefined dictionary, e.g. MedDRA, which is updated biannually with many new categories. The objective of this paper is to study interobserver variation and other challenges of coding.
MethodsSystematic review using PRISMA. We searched PubMed, EMBASE and The Cochrane Library. All studies were screened for eligibility by two authors.
ResultsOur search returned 520 unique studies of which 12 were included. Only one study investigated interobserver variation. It reported that 12% of the codes were evaluated differently by two coders. Independent physicians found that 8% of all the codes deviated from the original description. Other studies found that product summaries could be greatly affected by the choice of dictionary. With the introduction of MedDRA, it seems to have become harder to identify adverse events statistically because each code is divided in subgroups. To account for this, lumping techniques have been developed but are rarely used, and guidance on when to use them is vague. An additional challenge is that adverse events are censored if they already occurred in the run-in period of a trial. As there are more than 26 ways of determining whether an event has already occurred, this can lead to bias, particularly because data analysis is rarely performed blindly.
ConclusionThere is a lack of evidence that coding of adverse events is a reliable, unbiased and reproducible process. The increase in categories has made detecting adverse events harder, potentially compromising safety. It is crucial that readers of medical publications are aware of these challenges. Comprehensive interobserver studies are needed.

背景：临床试验中不良事件（adverse events）的分类错误有时会造成严重后果。因此，从患者报告不良事件到最终在出版物表格中呈现的全流程各环节，均应尽可能实现标准化，最大程度减少主观解读空间。不良事件需通过预定义词典进行分类，例如监管活动医学词典（MedDRA），该词典每两年更新一次，新增大量分类条目。本研究旨在探讨编码工作中的观察者间变异（interobserver variation）及其他相关挑战。

方法：采用系统评价方法，遵循系统评价与荟萃分析优先报告条目（PRISMA）规范。检索范围涵盖PubMed数据库、EMBASE数据库及Cochrane图书馆。由两名研究者独立对所有研究进行纳入资格筛选。

结果：本次检索共获得520篇独立研究，最终纳入12篇。仅1项研究探讨了观察者间变异问题，该研究显示，12%的编码结果在两名编码者间存在评估差异。独立医师评估发现，8%的编码结果与原始描述不符。另有研究表明，词典的选择会对研究结果总结产生显著影响。随着MedDRA的引入，由于每个编码均被细分为亚组，统计学上识别不良事件的难度有所增加。为此，学界已开发出合并编码技术（lumping techniques），但该技术的应用率极低，且关于其适用时机的指导原则较为模糊。此外，若不良事件已发生于临床试验的导入期（run-in period），则会被截尾剔除。由于判断事件是否已发生的方法超过26种，这可能导致偏倚，尤其是在数据分析极少采用盲法的情况下。

结论：目前尚无证据表明不良事件编码是一项可靠、无偏倚且可重复的流程。分类条目数量的增加使得不良事件的识别难度上升，可能危及临床试验安全性。医学出版物的读者亟需了解此类挑战，开展全面的观察者间变异研究实属必要。