PeerRead v1

PeerRead v1是由卡内基梅隆大学和艾伦人工智能研究所共同创建的第一个公开科学同行评审数据集，包含14.7K篇论文草稿及其接受/拒绝决定，以及10.7K篇专家撰写的文本同行评审。数据集涵盖了ACL、NIPS和ICLR等顶级会议的论文。创建过程中，研究团队与会议主席和会议管理系统合作，允许作者和评审者选择加入他们的论文草稿和评审。此外，还爬取了公开的同行评审，并对文本评审进行了数值评分标注。该数据集可用于分析同行评审过程的细微差别，如评审的一致性、偏见等，并可用于NLP任务，如预测论文接受情况和评审方面评分，旨在帮助改进科学出版过程。

PeerRead v1 is the first open scientific peer review dataset jointly created by Carnegie Mellon University and the Allen Institute for AI. It contains 14.7K paper drafts paired with their acceptance or rejection decisions, and 10.7K textual peer reviews authored by domain experts. The dataset covers papers from top academic conferences including ACL, NIPS and ICLR. During its creation, the research team collaborated with conference chairs and conference management systems, granting authors and reviewers the option to opt in their submitted paper drafts and associated reviews. Additionally, public peer reviews were crawled, and numerical score annotations were assigned to the textual review contents. This dataset can be utilized to analyze the nuances of the peer review process, such as review consistency and reviewer bias, as well as to support NLP tasks including predicting paper acceptance outcomes and review scores, with the aim of helping improve the scientific publishing process.