NILK

A dataset for the NIL-detection and NIL-disambiguation tasks. The NILK dataset has two main features: 1) It marks NIL-mentions for NIL-detection by extracting mentions which belong to newly added entities in Wikipedia text. 2) It provides an entity label for NIL-disambiguation by marking NIL-mentions with WikiData IDs from the newer dump. Dataset files contain JSON objects of the following structure: <pre><code class="language-json">{"mention":"Walter Damrosch", "offset":348, "length":15, "context":"...the conductor Walter Damrosch. He scored the piece for the standard instruments of the symphony orchestra plus celesta, saxophone, and automobile horns...", "wikipedia_page_id":"309", "wikidata_id":"Q725579", "nil":false}</code></pre> The dataset contains both linked and not linked mentions, one can distinguish between them by checking "nil" flag. To obtain NIL-mentions, we compared two WikiData dumps: from 2017 and 2021. NIL-mentions have WikiData ID from WikiData 2021, one can use it to check whether these mentions refer to the same entity. The dataset was designed with the WikiData 2017 as the target knowledge base in mind: https://archive.org/download/wikibase-wikidatawiki-20170213/wikidata-20170213-all.json.gz nilk_03_2023.zip contains same data with longer contexts (unsplit)本数据集面向NIL检测（NIL-detection）与NIL消歧（NIL-disambiguation）任务。NILK数据集具备两大核心特性：1）针对NIL检测任务标记NIL提及（NIL-mentions），通过提取维基百科文本中属于新增实体的提及内容实现；2）为NIL消歧任务提供实体标签，通过为NIL提及标注来自新版维基数据（WikiData）转储文件的维基数据ID实现。 数据集文件采用如下结构的JSON对象存储： <pre><code class="language-json">{"mention":"Walter Damrosch", "offset":348, "length":15, "context":"...the conductor Walter Damrosch. He scored the piece for the standard instruments of the symphony orchestra plus celesta, saxophone, and automobile horns...", "wikipedia_page_id":"309", "wikidata_id":"Q725579", "nil":false}</code></pre> 本数据集同时涵盖已链接提及与未链接提及，可通过检查"nil"标记字段区分二者。为获取NIL提及样本，本数据集通过对比2017年与2021年的两版维基数据转储文件完成构建。NIL提及带有2021版维基数据ID，可通过该ID验证此类提及是否指向同一实体。本数据集以2017版维基数据作为目标知识库构建，相关资源下载地址为：https://archive.org/download/wikibase-wikidatawiki-20170213/wikidata-20170213-all.json.gz nilk_03_2023.zip 包含内容一致但上下文更长（未拆分）的数据集。