zhouruiyang/RR-MCQ

---
license: mit
task_categories:
- question-answering
---
# Dataset Card for Dataset Name

<!-- Provide a quick summary of the dataset. -->

RR-MCQ (Review-Rebuttal Multiple-Choice Question) is an evaluation dataset for models' reviewing-related abilities.

## Dataset Details

<!-- Provide a longer summary of what this dataset is. -->

Description:
- contains 196 multiple-choice questions with 1-4 correct answers;
- questions are based on the review-rebuttal forums of ICLR-2023 on Openreview;
- each question is generated from a related argument=(review, response);
- each question has 4 types of labels: review aspect, paper content, ability, need extra info.

Content:
- paper basic info: title, keywords, tl_dr, abstract, decision;
- argument pair (review, response);
- question, 4 options, answers;
- 4 labels.

## Citation

<!-- If there is a paper or blog post introducing the dataset, the APA and Bibtex information for that should go in this section. -->

Please cite the paper of LREC-COLING 2024 "Is LLM a Reliable Reviewer? A Comprehensive Evaluation of LLM on Automatic Paper Reviewing Tasks".

原始信息汇总

数据集卡片：RR-MCQ

数据集概述

RR-MCQ（Review-Rebuttal Multiple-Choice Question）是一个用于评估模型审阅相关能力的评估数据集。

数据集详情

描述

• 包含196个多选题，每个问题有1-4个正确答案；
• 问题基于ICLR-2023在Openreview上的审阅-反驳论坛；
• 每个问题由相关的论点（审阅，回应）生成；
• 每个问题有4种类型的标签：审阅方面、论文内容、能力、需要额外信息。

内容

• 论文基本信息：标题、关键词、tl_dr、摘要、决策；
• 论点对（审阅，回应）；
• 问题、4个选项、答案；
• 4种标签。

引用

请引用LREC-COLING 2024的论文《Is LLM a Reliable Reviewer? A Comprehensive Evaluation of LLM on Automatic Paper Reviewing Tasks》。