mocha

MOCHA旨在提供一个用于训练和评估生成式阅读理解指标的基准。它包含从6个不同的问答数据集中提取的4万个人工判断分数，以及用于评估的额外最小对。数据内容包括文章段落、问题、参考答案以及模型生成的候选答案，并附带人工评分。MOCHA支持训练用于阅读理解的评估指标，并提供训练、验证、测试和最小对等数据集划分。该数据集采用CC BY-SA 4.0授权许可。

MOCHA is a benchmark designed for training and evaluating generative reading comprehension metrics. It comprises 40,000 human-annotated scores extracted from six distinct question answering datasets, alongside additional minimal pairs for evaluation purposes. The dataset includes article passages, questions, reference answers, model-generated candidate answers, paired with corresponding human-rated scores. MOCHA supports the training of reading comprehension evaluation metrics, and provides dataset splits for training, validation, test, and minimal pair subsets. This dataset is licensed under CC BY-SA 4.0.