neogenesislab/korean-rag-ssot-golden-50

Korean RAG SSOT Golden 50 (Neo Genesis) 是一个韩语检索增强生成评估数据集，包含50个手工策划的任务，用于在真实世界的RAG代理上测试代理治理、自主交易和安全/PII编辑场景。该数据集从Neo Genesis的实时SSOT（单一真相源）存储库中提取，涵盖了设计文档、运行手册、代码、事件日志和政策YAML等内容，旨在评估从操作文档中检索答案的能力，而非百科全书式的事实QA。数据集包含五个任务类别和五个评估指标，每个任务都带有来源类型信息，以评估检索器是否正确信任人工撰写的块而非合成内容。Korean RAG SSOT Golden 50 (Neo Genesis) is a Korean-language retrieval-augmented generation evaluation set of 50 hand-curated tasks built by Neo Genesis for stress-testing real-world RAG agents on agent governance, autonomous trading, and security/PII redaction scenarios. The dataset was extracted from Neo Genesis live SSOT (Single Source of Truth) repository, which includes design docs, runbooks, code, incident logs, and policy YAML, focusing on evaluating the ability to retrieve answers from operational documents rather than encyclopedic facts. It features five task categories and five evaluation metrics, with each task carrying provenance information to assess whether retrievers correctly trust human-authored chunks over synthetic ones.