ZDZR/BRINK-FB15k-237

---
pretty_name: BRINK-FB15k-237
task_categories:
- question-answering
- text-generation
language:
- en
tags:
- knowledge-graph
- rag
- reasoning
- benchmark
- kgqa
- incomplete-knowledge
license: mit
size_categories:
- unknown
---

# BRINK-FB15k-237

BRINK (Benchmark for Reasoning under Incomplete Knowledge) is a benchmark for evaluating Knowledge Graph–based Retrieval-Augmented Generation (KG-RAG) under incomplete knowledge. Unlike standard KGQA benchmarks, BRINK is designed so that each question cannot be answered by directly retrieving a single explicit supporting triple. Instead, the answer must be inferred from alternative reasoning paths that remain in the graph after the directly supporting fact is removed.

BRINK-FB15k-237 is the FB15k-237 split of BRINK, providing a more realistic benchmark setting for evaluating reasoning robustness under missing direct evidence. This repository may include variants with textual labels and/or anonymized entity identifiers.

🌐 **Website:** https://github.com/boschresearch/brink

📄 **Paper:** https://arxiv.org/abs/2508.08344

## Construction

BRINK is constructed by mining high-confidence logical rules from the original knowledge graph and using rule groundings to generate question-answer pairs. For each instance, the directly supporting triple is removed while preserving sufficient alternative evidence for inference.

## Intended Use

This dataset is intended for benchmarking KGQA and KG-RAG methods on medium-scale knowledge graphs where direct answer facts are unavailable and reasoning is required.

## Citation

If you use this dataset, please cite:

```bibtex
@inproceedings{zhou2026breaks,
title={What Breaks Knowledge Graph based RAG? Benchmarking and Empirical Insights into Reasoning under Incomplete Knowledge},
author={Zhou, Dongzhuoran and Zhu, Yuqicheng and Wang, Xiaxia and Zhou, Hongkuan and He, Yuan and Chen, Jiaoyan and Staab, Steffen and Kharlamov, Evgeny},
booktitle={The 19th Conference of the European Chapter of the Association for Computational Linguistics: EACL},
year={2026}
}