Non-inferiority and superiority analysis.

BackgroundThe overloading of health care systems is an international problem. In this context, new tools such as symptom checker (SC) are emerging to improve patient orientation and triage. This SC should be rigorously evaluated and we can take a cue from the way we evaluate medical students, using objective structured clinical examinations (OSCE) with simulated patients.ObjectiveThe main objective of this study was to evaluate the efficiency of a symptom checker versus emergency physicians using OSCEs as an assessment method.MethodsWe explored a method to evaluate the ability to set a diagnosis and evaluate the emergency of a situation with simulation. A panel of medical experts wrote 220 simulated patients cases. Each situation was played twice by an actor trained to the role: once for the SC, then for an emergency physician. Like a teleconsultation, only the patient’s voice was accessible. We performed a prospective non-inferiority study. If primary analysis had failed to detect non-inferiority, we have planned a superiority analysis.ResultsThe SC established only 30% of the main diagnosis as the emergency physician found 81% of these. The emergency physician was also superior compared to the SC in the suggestion of secondary diagnosis (92% versus 52%). In the matter of patient triage (vital emergency or not), there is still a medical superiority (96% versus 71%). We prove a non-inferiority of the SC compared to the physician in terms of interviewing time.Conclusions and relevanceWe should use simulated patients instead of clinical cases in order to evaluate the effectiveness of SCs.背景 医疗保健系统过载是一个全球性问题。在此背景下，症状检查器（symptom checker，SC）等新型工具应运而生，旨在优化患者就医导向与分诊流程。此类症状检查器需经过严格评估，我们可借鉴医学生的评估范式，采用结合模拟患者的客观结构化临床考试（objective structured clinical examinations，OSCE）开展评估工作。 研究目的 本研究的核心目标是以客观结构化临床考试作为评估方法，对比症状检查器与急诊医师的临床诊疗效能。 研究方法 本研究探索了一种通过模拟场景评估诊断能力与急诊情境研判能力的评估方案。由医学专家团队编写了220例模拟患者病例，每名病例均由经过角色训练的演员进行两次演绎：一次用于症状检查器测试，另一次用于急诊医师评估。与远程问诊场景一致，仅可获取患者的语音信息。本研究为前瞻性非劣效性研究，若主要分析未检出非劣效性，则计划开展优效性分析。 研究结果 症状检查器确诊主要诊断的比例仅为30%，而急诊医师的确诊比例达81%。在次要诊断建议方面，急诊医师同样优于症状检查器（92% vs 52%）。在患者分诊（判断是否为急重症）环节，急诊医师仍展现出更高的准确性（96% vs 71%）。本研究证实，症状检查器在问诊时长方面与急诊医师无显著差异，符合非劣效性标准。 结论与意义 我们应采用模拟患者而非传统临床病例，来评估症状检查器的临床效能。