Supplementary Material for: Critical appraisal tool for homeopathic intervention studies – CATHIS

Background: The quality of homeopathic research studies is controversially discussed. In order to improve the overall assessment of homeopathic studies and subsequently facilitate evidence-based clinical decision-making, we aimed to combine methodological tools for scientific rigor with tools for external and model validity into one global tool.
Method: Against the background of previous literature and the results of a preparatory survey, a first draft of a potential overarching tool was presented to an expert panel. Then, Delphi feedback rounds were conducted to elicit responses on the feasibility. The resulting preliminary tool was pilot-tested by five research experts on five randomly selected studies. After further optimization another five studies were assessed by the same experts and another three experts tested the second version. The ratings were tested for interrater-reliability using Gwet’s AC2.
Results: The appraisal tool comprises four domains. These are the assessment of risk-of-bias (1) using the Cochrane Tool, the studies’ credibility (2) including the CONSORT statement and its extension on homeopathy, the study interventions’ coherence (3) with homeopathic textbooks and previous research, and of the studies’ clinical relevance (4). Each domain is assessed using a point system, which is then added up to an overall score. The instrument was evaluated as useful and “easy to apply” by the raters. The interrater reliability was 0.81 (95% CI 0.75 to 0.88) for five raters, and 0.64 (95% CI 0.49 to 0.79) for additional three raters in round 2, signifying good to moderate interrater reliability.
Conclusion: We have developed an applicable and reliable assessment tool for homeopathic intervention studies. Using it in systematic reviews and meta-analyses will increase their scientific value.