Implementation and evaluation of a multilingual search pilot in the Europeana digital library (dataset)

The dataset contains the data required to reproduce the experiments done in the paper "Implementation and evaluation of a multilingual search pilot in the Europeana digital library", published in the 26th International Conference on Theory and Practice of Digital Libraries (TPDL'22). In that work we implemented a pilot applying query translation to English from the Spanish version of the website in order to surface results that have English metadata associated with them. The dataset is also available at https://rnd-2.eanadev.org/share/crosslingual_SpanishPilot/, and it is organized in three main folders:

sample: stratified sample of 300 queries queries issued from the Europeana Spanish portal from 1st
December 2020 to 28th February 2021.

evaluation.translations: manual annotation of the quality of the identification of the language of the queries using Google Cloud Translation API, and the quality of the translation obtained using Google plus the CEF translation service (eTranslation).

evaluation.search_retrieval: manual annotation of the relevancy of the (binary) relevance of the documents that are retrieved by one system but not by the other (current monolingual version vs pilot) in their top ten.