TREC DL 2023 passage ranking

该数据集包含了人类提出的以及合成生成的查询问题，并附有人类专家和大型语言模型（LLM）的评估注释。这些查询问题总数为82条，其中包括51条由人类提出的真实问题，18条由GPT-4生成的问题，以及13条由T5生成的问题。评估采用了4点相关度量表，数据集中包含了1830条完全相关（3分），2259条高度相关（2分），4372条相关（1分），以及13866条不相关评估。这82条查询问题具有多个相关性评估，该数据集的任务是信息检索评估。

This dataset comprises human-posed authentic query questions and synthetically generated query questions, paired with evaluation annotations from human experts and Large Language Models (LLMs). In total, this dataset includes 82 unique query questions, which can be broken down into three categories: 51 authentic questions raised by humans, 18 questions generated by GPT-4, and 13 questions generated by T5. The evaluation employs a 4-point relevance rating scale, with the dataset containing 1830 entries labeled as fully relevant (score 3), 2259 entries labeled as highly relevant (score 2), 4372 entries labeled as relevant (score 1), and 13866 entries labeled as irrelevant. Each of the 82 query questions has multiple relevance evaluation results, and the core task supported by this dataset is information retrieval evaluation.