AdversarialQA

我们创建了三个使用对抗模型在环构建的新阅读理解数据集。我们使用三种不同的模型； BiDAF (Seo et al., 2016)、BERTLarge (Devlin et al., 2018) 和 RoBERTaLarge (Liu et al., 2019) 在标注循环中构建三个数据集； D(BiDAF)、D(BERT) 和 D(RoBERTa)，每个都有 10,000 个训练示例、1,000 个验证和 1,000 个测试示例。对抗性人类注释范式确保这些数据集包含当前最先进的模型（至少是在注释循环中用作对手的模型）认为具有挑战性的问题。三个 AdversarialQA 第 1 轮数据集为此类方法提供了培训和评估资源。

We present three novel reading comprehension datasets constructed via an adversarial model-in-the-loop annotation framework. We employ three distinct models—BiDAF (Seo et al., 2016), BERT_Large (Devlin et al., 2018), and RoBERTa_Large (Liu et al., 2019)—as the adversarial models in the annotation loop to build three datasets: D(BiDAF), D(BERT), and D(RoBERTa). Each dataset contains 10,000 training examples, 1,000 validation examples, and 1,000 test examples. The adversarial human annotation paradigm ensures that these datasets include questions deemed challenging by state-of-the-art models. The three AdversarialQA Round 1 datasets provide training and evaluation resources for such research methodologies.

搜集汇总

背景与挑战

背景概述

AdversarialQA是一个机器阅读理解数据集，包含三个基于不同模型（BiDAF、BERTLarge、RoBERTaLarge）构建的子集，每个子集包含10,000训练、1,000验证和1,000测试示例。该数据集通过对抗性注释范式设计，旨在挑战当前最先进的模型。