Hallu-TCM

We develope a novel TCM hallucination detection dataset, Hallu-TCM, sine no prior work has attempted this task in TM. We selected 1,260 TCM exam questions including 16 TCM subjects, input them into GPT-4, and collected their feedback. In the first level, we utilize Qwen-Max interface to annotate feedback multiple times with the binary label. If Qwen-Max consistently provided the same label across annotations, we adopted that label. For contentious cases, we recruited higher-degree research students who can understand and solve complex questions, including three Ph.D. students and one master's student. They can perform the secondary annotation by using any available tools to assist them. Finally, one TCM physician annotated any controversial feedback from the students' annotations to make the final decision.我们开发了一款全新的中医（Traditional Chinese Medicine，TCM）幻觉检测数据集Hallu-TCM，鉴于此前尚无针对中医领域该项任务的相关研究工作。我们筛选了涵盖16门中医学科的1260道中医考试题，将其输入GPT-4模型并收集其生成的反馈内容。在首轮标注阶段，我们通过通义千问Max（Qwen-Max）接口对反馈内容进行多次二元标签标注。若通义千问Max在多次标注中均输出相同标签，则采用该标签作为对应结果。对于存在分歧的标注案例，我们招募了具备复杂问题理解与解决能力的高层次科研学生，其中包括3名博士研究生与1名硕士研究生。他们可借助任意可用工具开展二次标注工作。最终，由一名中医医师对学生标注阶段产生的所有争议性反馈内容进行最终标注，以敲定最终标签。

搜集汇总

背景与挑战

背景概述

Hallu-TCM是一个专门用于检测传统中医（TCM）领域大语言模型幻觉的数据集，包含1,260个涵盖16个TCM科目的考试问题及其GPT-4生成的反馈，通过多级人工与自动标注流程确保数据质量，支持二元分类任务以提升AI在TCM中的可靠性。