Wikidata Property Ranking

A set of preference judgements among generated random property pairs for 350 random Wikidata persons. For each (entity, property1, property2) record, 10 annotators judged which of the two properties is more interesting for the respective entity.

The goal is then to predict the annotator judgments as good as possible.

Current state-of-the-art methods (Wikidata Property Suggester and others) achieve 61% precision in this task, while methods based on linguistic similarity get to 74%, still significantly below annotator agreement (87.5%).

Further details are in the paper "Doctoral Advisor or Medical Condition: Towards Entity-specific Rankings of Knowledge Base Properties", ADMA 2017, available at http://www.simonrazniewski.com/2017_ADMA.pdf

本数据集包含对350个随机选取的维基数据人物所生成的随机属性对的偏好判断。对于每个（实体，属性1，属性2）记录，10位标注者评判了两个属性中哪一个对于相应实体更为有趣。目标则是尽可能准确地预测标注者的判断。目前最先进的方法（如维基数据属性建议器等）在此任务中实现了61%的精确度，而基于语言相似度的方法达到了74%，但仍显著低于标注者的一致性（87.5%）。更多详细信息可参考论文《博士导师或医疗状况：迈向知识库属性针对实体的排名》，发表于ADMA 2017，可在http://www.simonrazniewski.com/2017_ADMA.pdf找到。