Hallu-TCM

We develope a novel TCM hallucination detection dataset, Hallu-TCM, sine no prior work has attempted this task in TM. We selected 1,260 TCM exam questions including 16 TCM subjects, input them into GPT-4, and collected their feedback. In the first level, we utilize Qwen-Max interface to annotate feedback multiple times with the binary label. If Qwen-Max consistently provided the same label across annotations, we adopted that label. For contentious cases, we recruited higher-degree research students who can understand and solve complex questions, including three Ph.D. students and one master's student. They can perform the secondary annotation by using any available tools to assist them. Finally, one TCM physician annotated any controversial feedback from the students' annotations to make the final decision.

本研究团队开发了一款新颖的中药材幻觉检测数据集，即 Hallu-TCM，鉴于此前尚无相关工作尝试在中药材领域开展此任务。我们选取了包含16个中药材科目的1260道中药材考试题目，将其输入至GPT-4模型，并收集其反馈。在第一层标注中，我们利用Qwen-Max界面对反馈进行多次二元标签标注。若Qwen-Max在多次标注中始终保持一致的标签，则采纳该标签。对于存在争议的情况，我们招募了能够理解和解决复杂问题的更高学位的研究生，包括三名博士研究生和一名硕士研究生。他们可利用任何可用工具进行二级标注。最终，由一位中药材医师对学生的标注中存在争议的反馈进行标注，以作出最终决定。