Social IQA (Social Interaction QA)

我们介绍了Social IQa：Social Interaction QA，一种用于测试社会常识智力的新问答基准。与许多先前关注物理或分类知识的基准相反，Social IQa 侧重于推理人们的行为及其社会影响。例如，给定“杰西看了一场音乐会”这样的动作和“杰西为什么要这样做？”这样的问题，人类可以很容易地推断出杰西想要“看到他们最喜欢的表演者”或“欣赏音乐”，而不是“看看里面发生了什么”或“看看它是否有效”。 Social IQa 中的动作跨越了各种各样的社会情境，候选答案包含人工策划的答案和经过对抗过滤的机器生成的候选答案。 Social IQa 包含超过 37,000 个 QA 对，用于评估模型推理日常事件和情况的社会影响的能力。

We introduce Social IQa: Social Interaction QA, a novel question-answering benchmark for testing social commonsense intelligence. In contrast to many prior benchmarks that focus on physical or taxonomic knowledge, Social IQa focuses on reasoning about people’s actions and their social impacts. For instance, given an action like "Jesse attended a concert" and a question like "Why did Jesse do this?", humans can easily infer that Jesse intended to "see their favorite performer" or "enjoy the music", rather than "see what was happening inside" or "see if it works". The actions in Social IQa span a wide range of social scenarios, and the candidate answer set includes both human-curated answers and adversarially filtered machine-generated candidates. Social IQa contains over 37,000 QA pairs for evaluating models' ability to reason about the social impacts of everyday events and situations.

搜集汇总

背景与挑战

背景概述

Social IQA是一个社会互动问答基准数据集，旨在评估模型对社会常识的推理能力，特别是人们行为及其社会影响的理解。该数据集包含超过37,000个问答对，覆盖多种社会情境，通过人工和对抗过滤的机器生成候选答案来构建。