LongEval 2025 CORE Retrieval Test Collection

This collection consists of queries and documents extracted from the CORE scholarly literature search engine (https://core.ac.uk/). The queries in this collection were issued by CORE users, and a specific pipeline was designed to capture user queries, their corresponding search results, and user interactions. Using this pipeline, the dataset for this task was extracted, consisting of two main components: 

Search Information – includes unique (anonymous) identifiers for individual user sessions, search queries, and the returned results. 

Click Information – includes unique (anonymous) identifiers for individual user sessions, the links clicked within the results list, and the positions of the clicked links. 

The documents in the collection were selected based on the search results of user queries. In addition to these selected documents, the collection also contains randomly chosen documents from the CORE index. 

The test data was collected in January 2025. In total, the collection comprises 99 queries which were held out from the training queries (November 2024) and 492 test queries fopr the test data in January 2025. The document set consists of 124,045 downloaded, cleaned, and filtered scholarly articles. 

This collection serves as the official testing dataset for the 2025 LongEval Information Retrieval Lab (https://clef-longeval.github.io/), organized at CLEF. It includes the relevance assessments for the test queries.