Datasets for Out-of-KB Mention Discovery with Entity Linking

The repository contains datasets for out-of-KB mention discovery from texts, documented in the work, Reveal the Unknown: Out-of-Knowledge-Base Mention Discovery with Entity Linking, on arXiv: https://arxiv.org/abs/2302.07189 (CIKM 2023).

Each data setting (as a sub-folder) contains train, valid, and test files and also 100 random sample files for each data split for debugging.

Data folder names with “syn_full” at the end are synonym augmented data (each synonym as an entity) for the setting.

Ontology .jsonl files have two versions for each, "syn_attr" setting treats synonyms are attributes, "syn_full" setting treats synonyms as entities.

Data scripts are available at https://github.com/KRR-Oxford/BLINKout#data-scripts

Acknowledgement of the data sources below:

ShARe/CLEF 2013 dataset is from https://physionet.org/content/shareclefehealth2013/1.0/

MedMention dataset is from https://github.com/chanzuckerberg/MedMentions

UMLS (versions 2012AB, 2014AB, 2017AA) is from https://www.nlm.nih.gov/research/umls/index.html

SNOMED CT (corresponding versions) is from https://www.nlm.nih.gov/healthit/snomedct/index.html

NILK dataset is from https://zenodo.org/record/6607514

WikiData 2017 dump is from https://archive.org/download/enwiki-20170220/enwiki-20170220-pages-articles.xml.bz2