Language Specific Event Recommendation Ground Truth

This is a multilingual ground truth dataset for training, evaluating and testing the LaSER (Language-Specific Event Recommendation) model. It contains language-specific relevance scores for event-centric click-through pairs according to the publicly available Clickstream dataset in German, French and Russian as well as the user study annotations conducted for evaluating the language-specific recommendations by LaSER. For more details, refer to EventKG+Click and LaSER.

This dataset consists of two sets of files as follows:
1. The ground truth dataset that is used for training the learning to rank (LTR) model in LaSER in three languages. The following files contain the language-specific relevance scores between a source and target entity based on EventKG+Click dataset:

german_ground_truth.txt

french_ground_truth.txt

russian_ground_truth.txt

In these files source and target represent the label of entities and events in the respective language. 

2. The second set contains the user study participants' annotations regarding different relevance criteria of recommended events by LaSER. The following three files contain the annotations of at least three participants per event:

german_user_study_annotations.csv

french_user_study_annotations.csv

russian_user_study_annotations.csv

In these files, "r1", "r2" and "r3" denote relevance to the topic, language community and general audience respectively. And topic and event represent the wikidata-id of entities and events.