DiscoSense

我们提出DiscoSense，这是通过理解各种话语连接词来进行常识性推理的基准。我们使用条件对抗性过滤 (一种使用条件生成的对抗性过滤的扩展) 在DiscoSense中生成引人注目的干扰物。我们证明了最先进的预先训练的语言模型很难在DiscoSense上表现良好，这使得该数据集非常适合评估下一代常识推理系统。

We propose DiscoSense, a benchmark for commonsense reasoning through the understanding of various discourse connectives. We employ conditional adversarial filtering, an extension of adversarial filtering that leverages conditional generation, to generate compelling distractors for DiscoSense. We demonstrate that state-of-the-art pre-trained language models struggle to achieve strong performance on DiscoSense, rendering this dataset highly suitable for evaluating next-generation commonsense reasoning systems.

搜集汇总

背景与挑战

背景概述

DiscoSense是一个通过理解话语连接词进行常识性推理的基准数据集，采用条件对抗性过滤技术生成干扰物，适合评估下一代常识推理系统的性能。该数据集由德克萨斯大学达拉斯分校于2022年发布。