lrsbrgrn/HalluGuard-Preferences-76k
收藏资源简介：
--- license: apache-2.0 task_categories: - text-classification language: - en tags: - reasoning - preferences - hallucination-detection - data-reformation - synthetic - orpo - rag size_categories: - 10K<n<100K --- <p align="center"> <img src="https://github.com/lrsbrgrn/blogging-resources/blob/main/HalluGuard/halluguard-prefs.png?raw=true" alt="HalluClaim" width="700"/> </p> <div align="center"> <h1>🛡️ HalluClaim-Prefs: A 76K Synthetic Preference Dataset for Document-Grounded Hallucination Detection</h1> </div> ## 🌍 Overview This dataset was used to fine-tune [HalluGuard-Qwen3-4B](https://huggingface.co/lrsbrgrn/HalluGuard-Qwen3-4B) via Odds Ratio Preference Optimization (ORPO). It consists of 76,708 high-quality preference tuples designed to teach the model how to reason and justify its hallucination detection. ## 📖 Publication This dataset was introduced in our paper at the 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026). > *HalluGuard: Evidence-Grounded Small Reasoning Models to Mitigate Hallucinations in Retrieval-Augmented Generation* ## 🏗️ Dataset Construction (Chosen vs. Rejected) The dataset uses a "Size-Based Heuristic" to create pairs. Every example contains a *Prompt* (the task), a *Chosen* response (the good behavior), and a *Rejected* response (the bad behavior). ### 1. The David vs. Goliath Setup * **Chosen Response:** Generated by ([Qwen/Qwen3-235B-A22B](https://huggingface.co/Qwen/Qwen3-235B-A22B)). These responses represent high-quality reasoning and correct labels. * **Rejected Response:** Generated by ([Qwen/Qwen3-0.6B](https://huggingface.co/Qwen/Qwen3-0.6B)). These responses represent weaker reasoning, lack of evidence, or incorrect labels. ### 2. Multi-Stage Filtering To ensure the *Chosen* response is actually superior, we applied: * **Label Verification:** We discarded the pair if the Qwen3-235B-A22B's classification didn't match the original ground-truth label. * **Consensus Filtering:** Two independent "Judge" models ([openai/gpt-oss-120b](https://huggingface.co/openai/gpt-oss-120b) and [DeepSeek-V3.1](https://huggingface.co/deepseek-ai/DeepSeek-V3.1-Terminus)) reviewed the pairs. Only pairs where both judges preferred the Qwen3-235B-A22B's answer were kept. ## 📦 Dataset Structure | Field | Content | | :--- | :--- | | **Prompt** | Instructions + Document + Claim | | **Chosen** | `<think>Detailed reasoning</think>` + `<answer><classification>Label</classification><justification>Evidence</justification></answer>` | | **Rejected** | `<think>Flawed/Short reasoning</think>` + `<answer><classification>Label</classification><justification>Evidence</justification></answer>` | ## ⚠️ Limitations - Synthetic data: may not capture all real-world hallucination patterns - English-only - Document-grounded: labels reflect support relative to the document, not real-world truth - Potential biases inherited from web data (i.e., FineWeb) and style rewriting ## ⚖️ Ethical Considerations Models trained on HalluClaim should be used as decision-support systems, not fully autonomous systems. Errors may include: - false positives (flagging correct claims) - false negatives (missing hallucinations) Human oversight is recommended, especially in sensitive domains. ## 📚 Citation ```bibtex @article{bergeron2025halluguard, title={HalluGuard: Evidence-Grounded Small Reasoning Models to Mitigate Hallucinations in Retrieval-Augmented Generation}, author={Bergeron, Loris and Buhnila, Ioana and François, Jérôme and State, Radu}, journal={arXiv preprint arXiv:2510.00880}, year={2025} } ```
许可证: Apache 2.0 任务类别: - 文本分类（text-classification） 语言: - 英语（en） 标签: - 推理（reasoning） - 偏好（preferences） - 幻觉检测（hallucination-detection） - 数据重构（data-reformation） - 合成（synthetic） - ORPO - 检索增强生成（Retrieval-Augmented Generation, RAG） 样本规模: - 10K<n<100K --- <p align="center"> <img src="https://github.com/lrsbrgrn/blogging-resources/blob/main/HalluGuard/halluguard-prefs.png?raw=true" alt="HalluClaim" width="700"/> </p> <div align="center"> <h1>🛡️ HalluClaim-Prefs：面向文档锚定（Document-Grounded）幻觉检测的7.6万条合成偏好数据集</h1> </div> ## 🌍 概述 本数据集用于通过优势比偏好优化（Odds Ratio Preference Optimization, ORPO）对HalluGuard-Qwen3-4B进行微调。该数据集包含76708条高质量偏好元组，旨在教会模型如何进行推理并为其幻觉检测结果提供依据。 ## 📖 发表信息 本数据集已在第64届国际计算语言学协会年会（ACL 2026）的论文中提出。论文标题为《HalluGuard：面向检索增强生成（Retrieval-Augmented Generation, RAG）中幻觉缓解的证据锚定小型推理模型》。 ## 🏗️ 数据集构建（优质样本vs劣质样本） 数据集采用“规模基准启发式算法”构建正负样本对。每个示例均包含任务提示（Prompt）、优质响应（Chosen）与劣质响应（Rejected）。 ### 1. 大卫与歌利亚式样本构建 * **优质响应（Chosen Response）**：由Qwen/Qwen3-235B-A22B生成，这类响应具备高质量推理过程与正确的标签标注。 * **劣质响应（Rejected Response）**：由Qwen/Qwen3-0.6B生成，这类响应存在推理薄弱、缺乏证据支撑或标签错误的问题。 ### 2. 多阶段过滤流程 为确保优质响应确实优于劣质响应，我们采用了以下过滤步骤： * **标签验证**：若Qwen3-235B-A22B的分类结果与原始真实标签（ground-truth label）不符，则丢弃该样本对。 * **共识过滤**：由两个独立的“评判模型”——openai/gpt-oss-120b与DeepSeek-V3.1-Terminus——对样本对进行评审，仅保留两个评判模型均偏好Qwen3-235B-A22B生成结果的样本对。 ## 📦 数据集结构 | 字段 | 内容 | | :--- | :--- | | **Prompt（提示）** | 任务指令 + 参考文档 + 待验证声明 | | **Chosen（优质响应）** | `<think>详细推理过程</think>` + `<answer><classification>标签</classification><justification>证据支撑</justification></answer>` | | **Rejected（劣质响应）** | `<think>存在缺陷/简略的推理过程</think>` + `<answer><classification>标签</classification><justification>证据支撑</justification></answer>` | ## ⚠️ 局限性 - 本数据集为合成数据，可能无法覆盖所有现实世界中的幻觉模式 - 仅支持英语语料 - 基于文档锚定：标签仅反映声明相对于参考文档的支撑性，而非现实世界中的真实情况 - 可能继承自网页数据（即FineWeb）与风格重写过程中引入的潜在偏差 ## ⚖️ 伦理考量 基于HalluClaim训练的模型仅应用作决策支持系统，而非完全自主的系统。该模型可能出现以下错误： - 假阳性：将正确的声明误判为存在幻觉 - 假阴性：未检测到实际存在的幻觉 建议在使用时引入人工审核，尤其是在敏感领域中。 ## 📚 引用 bibtex @article{bergeron2025halluguard, title={HalluGuard：面向检索增强生成中幻觉缓解的证据锚定小型推理模型}, author={Bergeron, Loris and Buhnila, Ioana and François, Jérôme and State, Radu}, journal={arXiv预印本 arXiv:2510.00880}, year={2025} }
