Document-to-document relevant assessment for TREC Genomics Track 2005

Here we present a table with document-to-document relevance assessment judgements on a subset of the TREC Genomics Track 2005 which corresponds to document-to-topic relevance assessments. This data was produced by four annotators to make it possible to analyze inter-annotator agreements as part of our future work. The data was produced with and in-house annotation tool tailored to the initial TREC data and the task at hand. The "raw data document evaluation" contains six columns, first row consecutive id, second original TREC topic, third PubMed Id used as reference document, fourth PMID used to evaluate the relevance wrt the reference document, fifth the relevance score (2 definitely relevant, 1 partially relevant, 0 non-relevant), and sixth annotator id.

Acknowledgements

This work is part of the STELLA project funded by DFG (project no. 407518790). This work was supported by the BMBF-funded de.NBI Cloud within the German Network for Bioinformatics Infrastructure (de.NBI) (031A532B, 031A533A, 031A533B, 031A534A, 031A535A, 031A537A, 031A537B, 031A537C, 031A537D, 031A538A).