OSKGC: A benchmark for Ontology Schema-based Knowledge Graph Construction from text

OSKGC is a benchmark dataset designed for knowledge graph construction based on a predefined ontology schema. Dataset Composition The OSKGC benchmark consists of three core components: Text-triple-schema data pairs Predefined ontology schemas A unified ontology hierarchy Based on WebNLG's text-triple pairs, we performed fine-grained type annotation for entities using external knowledge from DBpedia and Wikipedia, and constructed subsumption hierarchies according to DBpedia ontology classes. Text-Triple-Schema Pairs benchmark/data OSKGC utilizes text-triple pairs from WebNLG containing 1, 2, and 3 triples, totaling 10,183 text entries. These are divided into three groups by triple count, with each group further categorized into 19 thematic categories (57 total categories). Each text entry is unique, with entity types and relations overlapping across categories, with a total of 207 and 382 respectively. The following is an example entry: <entry category="3_Airport" id="3_Airport_train_28"> <text>Allama Iqbal International Airport serves the city of Lahore which is in Pakistan where Nawaz Sharif is the leader.</text> <triples> <triple> _Lahore <rel>country</rel> <obj>Pakistan</obj> </triple> <triple> _{Allama_Iqbal_International_Airport} <rel>serves</rel> <obj>Lahore</obj> </triple> <triple> _Pakistan <rel>leader</rel> <obj>Nawaz_Sharif</obj> </triple> </triples> <schemas> <schema> _City <rel>country</rel> <obj>Country</obj> </schema> <schema> _Airport <rel>serves</rel> <obj>City</obj> </schema> <schema> _Country <rel>leader</rel> <obj>PrimeMinister</obj> </schema> </schemas> </entry> <text>: contains textual information <triples>: defines a set of triples in (subject, relation, object) format <schemas>: defines corresponding ontology schemas in (subject type, relation, object type) format Alignment: Triples and schemas maintain strict one-to-one positional correspondence In our task formulation: Text serves as model input Triples and schemas serve as golden labels for: Triple-level factual accuracy evaluation Schema-level structural conformity evaluation Predefined Ontology Schemas benchmark/ontology For each category, a dedicated ontology schema is constructed to guide knowledge graph construction under the domain. These schemas contain entity types, their relations, and hierarchical structures within each category. An ontology schema consists of: Core Structure: Root-level entity types Category-specific relations Hierarchical Structure: Fine-grained entity types Connected to core structure via root nodes Hierarchical Structure benchmark/hierarchy.xml While relation structures vary by category, all hierarchies follow a standardized framework based on DBpedia ontology classes. This XML file provides the unified subsumption hierarchical structure used throughout OSKGC. For more details, please refer to our paper and GitHub repository.