MOCHA

MOCHA是由加州大学欧文分校和希伯来大学合作创建的数据集，旨在训练和评估生成式阅读理解度量标准。该数据集包含40,000个人类评分的模型输出，来自6个不同的问题回答数据集，以及一组额外的最小对用于评估。MOCHA通过收集大量的人类判断分数，训练了一个名为LERC的阅读理解评估度量，该度量能够模仿人类判断分数，显著优于现有度量。数据集的应用领域主要集中在提高生成式阅读理解模型的准确性和鲁棒性，解决现有度量在处理阅读理解细微差别时的不足。

MOCHA is a dataset co-created by the University of California, Irvine and the Hebrew University of Jerusalem, designed to train and evaluate generative reading comprehension metrics. This dataset contains 40,000 human-rated model outputs sourced from six distinct question answering datasets, alongside a supplementary set of minimal pairs for evaluation purposes. By collecting a large volume of human judgment scores, MOCHA enables the training of LERC, a reading comprehension evaluation metric that closely mimics human judgment scores and significantly outperforms existing state-of-the-art metrics. The primary application domains of this dataset focus on improving the accuracy and robustness of generative reading comprehension models, addressing the shortcomings of current metrics when handling the subtle nuances inherent in reading comprehension tasks.