ReCAM(SemEval-2021 Task 4: Reading Comprehension of Abstract Meaning)

任务我们共享的任务有三个子任务。子任务1和2侧重于评估机器学习模型在两个抽象定义方面的性能 (Spreen和Schulz，1966; Changizi，2008)，我们分别称之为不可感知性和非特异性。子任务3旨在为他们的关系提供一些见解。
·子任务1: ReCAM-不可感知性
具体词指的是我们可以用感官直接感知的事物，事件和属性 (Spreen和Schulz，1966; Coltheart 1981; Turney等人，2011)，例如甜甜圈，树木和红色。相反，抽象词指的是远离即时感知的想法和概念。例子包括客观、文化和经济。在子任务1中，要求参与系统对不可感知的概念进行抽象含义的阅读理解。
下面是一个例子。给定一段话和一个问题，您的模型需要从五个候选人中选择替换 @ placeholder的最佳候选人。
·子任务2: ReCAM-非特异性
子任务2侧重于不同类型的定义。与土拨鼠和鲸鱼等具体概念相比，脊椎动物等超属动物被认为更抽象 (Changizi，2008)。
·子任务3: ReCAM-交叉子任务3旨在提供更多关于抽象的两种观点的关系的见解，在这个子任务中，我们测试了一个系统的性能，该系统在一个定义上进行了训练，并在另一个定义上进行了评估。

Our shared tasks consist of three subtasks. Subtasks 1 and 2 focus on evaluating the performance of machine learning models on two abstractly defined constructs (Spreen & Schulz, 1966; Changizi, 2008), which we refer to as imperceptibility and nonspecificity respectively. Subtask 3 aims to provide additional insights into their relationship.
· Subtask 1: ReCAM – Imperceptibility
Concrete words refer to entities, events, and attributes that can be directly perceived through our sensory systems (Spreen & Schulz, 1966; Coltheart, 1981; Turney et al., 2011), such as donuts, trees, and the color red. In contrast, abstract words refer to ideas and concepts that are distant from immediate perceptual experience, with examples including objectivity, culture, and economy. In Subtask 1, participating systems are required to perform reading comprehension on the abstract meanings of imperceptible concepts.
An example is given below: Given a passage and a question, your model needs to select the optimal candidate from five options to replace the @ placeholder.
· Subtask 2: ReCAM – Nonspecificity
Subtask 2 focuses on distinct types of definitions. Compared to concrete concepts such as marmots and whales, superordinate animal terms like vertebrates are considered more abstract (Changizi, 2008).
· Subtask 3: ReCAM – Cross-Task
Subtask 3 aims to provide further insights into the relationship between the two perspectives on abstraction. In this subtask, we test the performance of a system trained on one definition and evaluated on the other.

搜集汇总

背景与挑战

背景概述

ReCAM是SemEval-2021 Task 4的数据集，专注于抽象含义的阅读理解，包含三个子任务：不可感知性、非特异性和交叉关系分析，旨在评估模型在不同抽象定义上的性能。该数据集由中国科学技术大学等机构于2021年发布，提供相关竞赛和论文链接。