PaDaS-Lab/CoRE

CoRE (控制检索评估)数据集是一个为严格评估信息检索中嵌入压缩技术而设计的基准数据集。它包含多种长度的段落和文档，以及从10k到100M不等的语料库大小。每个查询都有固定的相关和干扰文档数量，确保了不同语料库大小和文档类型下查询难度的一致性。数据集采用智能的语料库子采样策略构建，以保持固定的查询难度。

CoRE (Controlled Retrieval Evaluation) is a benchmark dataset designed for rigorous evaluation of embedding compression techniques in information retrieval. It includes various lengths of passages and documents, as well as corpus sizes ranging from 10k to 100M. Each query has a fixed number of relevant and distractor documents to ensure consistent query difficulty across different corpus sizes and document types. The dataset is constructed using an intelligent corpus subsampling strategy to maintain fixed query difficulty.