ZDZR/BRINK-FB15k-237

--- pretty_name: BRINK-FB15k-237 task_categories: - question-answering - text-generation language: - en tags: - knowledge-graph - rag - reasoning - benchmark - kgqa - incomplete-knowledge license: mit size_categories: - unknown --- # BRINK-FB15k-237 BRINK (Benchmark for Reasoning under Incomplete Knowledge) is a benchmark for evaluating Knowledge Graph–based Retrieval-Augmented Generation (KG-RAG) under incomplete knowledge. Unlike standard KGQA benchmarks, BRINK is designed so that each question cannot be answered by directly retrieving a single explicit supporting triple. Instead, the answer must be inferred from alternative reasoning paths that remain in the graph after the directly supporting fact is removed. BRINK-FB15k-237 is the FB15k-237 split of BRINK, providing a more realistic benchmark setting for evaluating reasoning robustness under missing direct evidence. This repository may include variants with textual labels and/or anonymized entity identifiers. 🌐 **Website:** https://github.com/boschresearch/brink 📄 **Paper:** https://arxiv.org/abs/2508.08344 ## Construction BRINK is constructed by mining high-confidence logical rules from the original knowledge graph and using rule groundings to generate question-answer pairs. For each instance, the directly supporting triple is removed while preserving sufficient alternative evidence for inference. ## Intended Use This dataset is intended for benchmarking KGQA and KG-RAG methods on medium-scale knowledge graphs where direct answer facts are unavailable and reasoning is required. ## Citation If you use this dataset, please cite: ```bibtex @inproceedings{zhou2026breaks, title={What Breaks Knowledge Graph based RAG? Benchmarking and Empirical Insights into Reasoning under Incomplete Knowledge}, author={Zhou, Dongzhuoran and Zhu, Yuqicheng and Wang, Xiaxia and Zhou, Hongkuan and He, Yuan and Chen, Jiaoyan and Staab, Steffen and Kharlamov, Evgeny}, booktitle={The 19th Conference of the European Chapter of the Association for Computational Linguistics: EACL}, year={2026} }--- pretty_name: BRINK-FB15k-237 task_categories: - 问答 - 文本生成 language: - 英语 tags: - 知识图谱（Knowledge Graph） - 检索增强生成（Retrieval-Augmented Generation, RAG） - 推理 - 基准测试 - KGQA - 不完备知识 license: MIT size_categories: - 未知 --- # BRINK-FB15k-237 BRINK（不完备知识下推理基准，Benchmark for Reasoning under Incomplete Knowledge）是一款用于评估不完备知识场景下基于知识图谱的检索增强生成（Knowledge Graph-based Retrieval-Augmented Generation, KG-RAG）的基准数据集。与标准知识图谱问答（Knowledge Graph Question Answering, KGQA）基准不同，BRINK的设计逻辑为：无法通过直接检索单条显式支持三元组（triple）来回答任一问题，作答必须通过移除直接支持事实后图中剩余的可选推理路径推导得出。 BRINK-FB15k-237是BRINK的FB15k-237拆分版本，为缺失直接证据场景下的推理鲁棒性评估提供了更贴合现实的基准设置。本仓库可能包含带有文本标签和/或匿名化实体标识符（entity identifiers）的变体版本。 🌐 **官方网站：** https://github.com/boschresearch/brink 📄 **论文：** https://arxiv.org/abs/2508.08344 ## 数据集构建 BRINK通过从原始知识图谱中挖掘高置信度逻辑规则，并利用规则接地（rule groundings）生成问答对构建而成。对于每个样本，直接支持三元组（triple）会被移除，同时保留足够的可选推理证据。 ## 预期用途 本数据集旨在针对无法获取直接答案事实、需要进行推理的中规模知识图谱，对KGQA与KG-RAG方法进行基准测试。 ## 引用声明 若您使用本数据集，请引用如下文献： bibtex @inproceedings{zhou2026breaks, title={What Breaks Knowledge Graph based RAG? Benchmarking and Empirical Insights into Reasoning under Incomplete Knowledge}, author={Zhou, Dongzhuoran and Zhu, Yuqicheng and Wang, Xiaxia and Zhou, Hongkuan and He, Yuan and Chen, Jiaoyan and Staab, Steffen and Kharlamov, Evgeny}, booktitle={The 19th Conference of the European Chapter of the Association for Computational Linguistics: EACL}, year={2026} }