RELISH-Aspire

This is a copy of the RELISH dataset used in the paper "Multi-Vector Models with Textual Guidance for Fine-Grained Scientific Document Similarity" by Sheshera Mysore, Arman Cohan, Tom Hope. The RELISH dataset was first introduced in Brown et al. 2019. <br><br>See further details of the paper, how this dataset was compiled, and how it was used: https://github.com/allenai/aspire<br>The contents of the dataset are as follows: <br><code>abstracts-relish.jsonl</code>: <code>jsonl</code> file containing the paper-id, abstracts, and titles for the queries and candidates which are part of the dataset.<br> <code>relish-queries-release.csv</code>: Metadata associated with every query.<code>test-pid2anns-relish.json</code>: JSON file with the query paper-id, candidate paper-ids for every query paper in the dataset. Use these files in conjunction with <code>abstracts-relish.jsonl</code> to generate files for use in model evaluation. <br><code>relish-evaluation_splits.json</code>: Paper-ids for the splits to use in reporting evaluation numbers. <code>aspire/src/evaluation/ranking_eval.py</code> included in the github repo accompanying this dataset implements the evaluation protocol and computes evaluation metrics. Please see the paper for descriptions of the experimental protocol we recommend to report evaluation metrics.<br>