Supplementary Material for: Critical appraisal tool for homeopathic intervention studies – CATHIS

Background: The quality of homeopathic research studies is controversially discussed. In order to improve the overall assessment of homeopathic studies and subsequently facilitate evidence-based clinical decision-making, we aimed to combine methodological tools for scientific rigor with tools for external and model validity into one global tool.
Method: Against the background of previous literature and the results of a preparatory survey, a first draft of a potential overarching tool was presented to an expert panel. Then, Delphi feedback rounds were conducted to elicit responses on the feasibility. The resulting preliminary tool was pilot-tested by five research experts on five randomly selected studies. After further optimization another five studies were assessed by the same experts and another three experts tested the second version. The ratings were tested for interrater-reliability using Gwet’s AC2.
Results: The appraisal tool comprises four domains. These are the assessment of risk-of-bias (1) using the Cochrane Tool, the studies’ credibility (2) including the CONSORT statement and its extension on homeopathy, the study interventions’ coherence (3) with homeopathic textbooks and previous research, and of the studies’ clinical relevance (4). Each domain is assessed using a point system, which is then added up to an overall score. The instrument was evaluated as useful and “easy to apply” by the raters. The interrater reliability was 0.81 (95% CI 0.75 to 0.88) for five raters, and 0.64 (95% CI 0.49 to 0.79) for additional three raters in round 2, signifying good to moderate interrater reliability.
Conclusion: We have developed an applicable and reliable assessment tool for homeopathic intervention studies. Using it in systematic reviews and meta-analyses will increase their scientific value.

背景：顺势疗法研究的质量长期以来备受争议。为优化顺势疗法研究的整体评估工作，进而推动循证临床决策的制定，本研究旨在将保障研究科学严谨性的方法学工具，与评估外部有效性及模型有效性的工具进行整合，打造一套通用全局评估工具。
研究方法：基于既往文献与预调查结果，研究团队将一套通用评估工具的初稿提交至专家论证小组。随后开展德尔菲法反馈轮次，征集关于工具可行性的专家意见。由此形成的初步评估工具，由5名研究专家对5篇随机遴选的研究文献开展预测试。经进一步优化完善后，原5名专家对另外5篇研究文献进行评估，另有3名新增专家对第二版工具开展测试。采用格威特AC2系数（Gwet’s AC2）对评分结果的评分者间信度进行检验。
研究结果：本评估工具包含4个维度：①采用Cochrane工具（Cochrane Tool）进行偏倚风险评估；②研究可信度维度，涵盖CONSORT声明（CONSORT statement）及其顺势疗法应用扩展版；③研究干预措施与顺势疗法教科书及既往研究的一致性维度；④研究临床相关性维度。各维度均采用计分规则进行评价，最终汇总得到总得分。参与测试的评分者认为该工具实用性强且‘易于操作’。首轮预测试中5名评分者的评分者间信度为0.81（95%置信区间：0.75~0.88）；第二轮测试中3名新增评分者的评分者间信度为0.64（95%置信区间：0.49~0.79），表明该工具具备良好至中等程度的评分者间信度。
结论：本研究开发了一套适用于顺势疗法干预性研究的评估工具，具备良好的实用性与可靠性。将该工具应用于系统评价与元分析（meta-analysis）中，可有效提升相关研究的科学价值。