SemEval annotated dataset

This dataset contains the annotations for 40 Latin lemmas taken from the diachronic Latin corpus LatinISE. The dataset was originally created as Latin test data for SemEval 2020 Task 1: Unsupervised Lexical Semantic Change Detection (Schlechtweg et al. 2020). The choice of the set of lexemes includes lexical units in which a change of meaning is observed in relation to Christianity and other socio-political changes in the late antiquity period. The annotation was performed manually: the annotators had to read each text snippet and assign the dictionary senses to each lemma in context on a graded scale, following a variation of the DuRel annotation framework (Schlechtweg et al. 2018): “1” indicated that the usage of the lemma in the text was unrelated to the dictionary sense, “2” indicated a distant relation between the two, “3” a close relation and “4” was used to indicate that the usage in the text completely overlapped with the dictionary sense. The label “0” was used when the annotator was not able to make a decision.

References

Schlechtweg, Dominik, im Walde Sabine Schulte & Stefanie Eckmann. 2018. Diachronic usage relatedness (DURel): A framework for the annotation of lexical semantic change. In Marilyn Walker, Heng Ji & Amanda Stent (eds.), The 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), 169–174. Stroudsburg, PA: Association for Computational Linguistics.

Schlechtweg, Dominik, Barbara McGillivray, Simon Hengchen, Haim Dubossarsky & Nina Tahmasebi. 2020. SemEval-2020 Task 1: Unsupervised lexical semantic change detection. In Aurelie Herbelot, Xiaodan Zhu, Alexis Palmer, Nathan Schneider, Jonathan May & Ekaterina Shutova (eds.), Proceedings of the fourteenth workshop on semantic evaluation, 1–23. Barcelona: International Committee for Computational Linguistics.