DLCC Gold Standard

Corresponding GitHub repository: DL-TC-Generator on GitHub

Abstract

Knowledge graph embedding is a representation learning technique which projects entities and relations in a knowledge graph to continuous vector spaces.
Embeddings have gained a lot of uptake and have been heavily used in link prediction and other downstream prediction tasks.
Most approaches are evaluated on a single task or a single group of tasks to determine their overall performance. The evaluation is then assessed in terms of how well the embedding approach performs on the task at hand, but it is hardly evaluated (and often not even deeply understood) what information the embedding approaches are actually learning to represent.

To fill this gap, we present the DLCC (Description Logic Class Constructors) benchmark, a resource to analyze embedding approaches in terms of which kinds of classes they can represent. Two gold standards are presented, one based on the real world knowledge graph DBpedia, and one synthetic gold standard.