WiRe57

WiRe57是由蒙特利尔大学创建的细粒度开放信息抽取基准，包含343个手动提取的事实，覆盖57个英文句子。数据集主要来源于Wikipedia和新闻文章，通过细致的标注过程，解决了指代消解和粒度问题，并尝试处理推理问题。该数据集旨在为开放信息抽取任务提供一个参考标准，用于评估和比较不同的开放信息抽取系统，特别是解决评估提取系统输出与参考元组对比的非平凡问题。

WiRe57 is a fine-grained open information extraction benchmark created by the University of Montreal. It contains 343 manually extracted facts spanning 57 English sentences. The dataset is primarily sourced from Wikipedia and news articles. Through a meticulous annotation process, it addresses coreference resolution and granularity issues, and attempts to handle reasoning-related problems. This benchmark aims to provide a reference standard for the open information extraction task, to evaluate and compare different open information extraction systems, particularly addressing the non-trivial problem of comparing the outputs of extraction systems against reference tuples.