NeuralPGRank/msmarco-hard-negatives

--- license: cc-by-sa-4.0 language: - en --- # Dataset Card ## Dataset Details This dataset contains a set of candidate documents for second-stage re-ranking on msmarco (dev, test split in [BEIR](https://huggingface.co/BeIR)). Those candidate documents are composed of hard negatives mined from [gtr-t5-xl](https://huggingface.co/sentence-transformers/gtr-t5-xl) as Stage 1 ranker and ground-truth documents that are known to be relevant to the query. This is a release from our paper [Policy-Gradient Training of Language Models for Ranking](https://gao-g.github.io/), so please cite it if using this dataset. ## Direct Use You can load the dataset by: ```python from datasets import load_dataset dataset = load_dataset("NeuralPGRank/msmarco-hard-negatives") ``` Each example is an dictionary: ```python >>> python dataset['test'][0] { "qid" : ..., # query ID "topk" : { doc ID: ..., # document ID as the key; None or a score as the value doc ID: ..., ... }, } ``` ## Citation ``` @inproceedings{Gao2023PolicyGradientTO, title={Policy-Gradient Training of Language Models for Ranking}, author={Ge Gao and Jonathan D. Chang and Claire Cardie and Kiant{\'e} Brantley and Thorsten Joachims}, booktitle={Conference on Neural Information Processing Systems (Foundation Models for Decising Making Workshop)}, year={2023}, url={https://arxiv.org/pdf/2310.04407} } ``` ## Dataset Card Author and Contact [Ge Gao](https://gao-g.github.io/)--- 许可证：CC-BY-SA-4.0 语言： - 英语 --- # 数据集卡片 ## 数据集详情 本数据集包含用于MSMARCO第二阶段重排序的候选文档集合，其开发集与测试划分集收录于[BEIR](https://huggingface.co/BeIR)。该候选文档集合由两部分构成：以[gtr-t5-xl](https://huggingface.co/sentence-transformers/gtr-t5-xl)作为第一阶段排序器挖掘得到的难负样本（hard negatives），以及已知与查询相关的真实标注文档（ground-truth documents）。本数据集源自我们的论文《用于排序的语言模型策略梯度训练》（[Policy-Gradient Training of Language Models for Ranking](https://gao-g.github.io/)），若使用本数据集请引用该论文。 ## 直接使用方式 你可以通过以下代码加载本数据集： python from datasets import load_dataset dataset = load_dataset("NeuralPGRank/msmarco-hard-negatives") 每个样本均为字典格式，示例如下： python >>> python dataset['test'][0] { "qid" : ..., # 查询ID "topk" : { 文档ID: ..., # 以文档ID作为键；值为None或排序分数 文档ID: ..., ... }, } ## 引用 @inproceedings{Gao2023PolicyGradientTO, title={Policy-Gradient Training of Language Models for Ranking}, author={Ge Gao and Jonathan D. Chang and Claire Cardie and Kianté Brantley and Thorsten Joachims}, booktitle={Conference on Neural Information Processing Systems (Foundation Models for Decising Making Workshop)}, year={2023}, url={https://arxiv.org/pdf/2310.04407} } ## 数据集卡片作者与联系方式 [Ge Gao](https://gao-g.github.io/)