RELISH-Aspire

This is a copy of the RELISH dataset used in the paper "Multi-Vector Models with Textual Guidance for Fine-Grained Scientific Document Similarity" by Sheshera Mysore, Arman Cohan, Tom Hope. The RELISH dataset was first introduced in Brown et al. 2019. <br><br>See further details of the paper, how this dataset was compiled, and how it was used: https://github.com/allenai/aspire<br>The contents of the dataset are as follows: <br><code>abstracts-relish.jsonl</code>: <code>jsonl</code> file containing the paper-id, abstracts, and titles for the queries and candidates which are part of the dataset.<br> <code>relish-queries-release.csv</code>: Metadata associated with every query.<code>test-pid2anns-relish.json</code>: JSON file with the query paper-id, candidate paper-ids for every query paper in the dataset. Use these files in conjunction with <code>abstracts-relish.jsonl</code> to generate files for use in model evaluation. <br><code>relish-evaluation_splits.json</code>: Paper-ids for the splits to use in reporting evaluation numbers. <code>aspire/src/evaluation/ranking_eval.py</code> included in the github repo accompanying this dataset implements the evaluation protocol and computes evaluation metrics. Please see the paper for descriptions of the experimental protocol we recommend to report evaluation metrics.<br>

本数据集为Sheshera Mysore、Arman Cohan与Tom Hope在论文《Multi-Vector Models with Textual Guidance for Fine-Grained Scientific Document Similarity》中所使用的RELISH数据集副本。RELISH数据集最早由Brown等人于2019年提出。

如需了解该论文的详细信息、数据集的构建流程与使用方式，请访问：https://github.com/allenai/aspire

本数据集包含以下文件：
1. `abstracts-relish.jsonl`：JSON Lines格式文件，存储数据集中查询与候选文献的论文ID、摘要及标题。
2. `relish-queries-release.csv`：对应每条查询的元数据文件。
3. `test-pid2anns-relish.json`：JSON格式文件，包含每条查询论文的ID及其对应的候选论文ID列表。需结合`abstracts-relish.jsonl`使用，以生成模型评估所需的文件。
4. `relish-evaluation_splits.json`：用于报告评估指标的数据集划分所对应的论文ID列表。

本数据集配套的GitHub仓库中提供的`aspire/src/evaluation/ranking_eval.py`实现了官方评估协议，并可计算各项评估指标。如需了解推荐的实验评估流程与指标说明，请参阅原论文。