AnnIE

AnnIE是一个交互式标注平台，旨在构建完整的开放信息抽取基准。该平台支持不同使用场景和语言，通过手动标注完整的事实（即同一事实的所有可接受表面实现集群）来更真实地衡量OIE系统的性能。AnnIE用于构建两个完整的OIE基准：一个涉及动词介导的事实，另一个涉及命名实体的事实。该平台通过公开发布，鼓励创建更多基于事实的OIE评估基准，而非基于标记级别的基准。

AnnIE is an interactive annotation platform designed to build comprehensive open information extraction (OIE) benchmarks. This platform supports diverse application scenarios and multiple languages, enabling more realistic performance evaluation of OIE systems by manually annotating complete facts, i.e., clusters of all acceptable surface realizations of the same fact. AnnIE is used to construct two complete OIE benchmarks: one focusing on verb-mediated facts and the other on facts related to named entities. Through its public release, this platform encourages the development of more fact-based OIE evaluation benchmarks rather than token-level benchmarks.