CODAH (COmmonsense Dataset Adversarially-authored by Humans)

COMmonsense Dataset Adversarially-authors by Humans (CODAH) 是一个评估集，用于 SWAG 句子完成风格的常识问答。与其他自动生成的 NLI 数据集不同，CODAH 是由人类对抗性构建的，他们可以查看来自预训练模型的反馈，并使用这些信息来设计具有挑战性的常识性问题。它总共包含 2801 个问题，并使用 5 折交叉验证进行评估。

The Commonsense Dataset Adversarially Authored by Humans (CODAH) is an evaluation benchmark for SWAG-style sentence completion common sense question answering. Unlike other automatically generated natural language inference (NLI) datasets, CODAH is adversarially constructed by human annotators who can access feedback from pre-trained models and leverage this information to design challenging commonsense questions. It contains a total of 2,801 questions, and is evaluated using 5-fold cross-validation.

搜集汇总

背景与挑战

背景概述

CODAH是一个由人类对抗性构建的常识问答评估集，采用SWAG句子完成风格，包含2801个问题并通过5折交叉验证进行评估。该数据集旨在设计具有挑战性的常识性问题，由西北大学于2019年发布。