SIQA (Social Interaction QA)

社交IQa: 社交互动QA，用于测试社交常识智能的新问答基准。与许多以前侧重于物理或分类学知识的基准相反，社会IQa侧重于对人们的行为及其社会影响的推理。例如，给定一个像 “杰西看到了一场音乐会” 这样的动作和一个像 “杰西为什么这样做？”，人类可以很容易地推断出杰西想要 “看他们最喜欢的表演者” 或 “欣赏音乐”，而不是 “看里面发生了什么” 或 “看它是否有效”。社交IQa中的动作跨越了各种各样的社交场合，并且答案候选者既包含人为策划的答案，也包含经过对手过滤的机器生成的候选者。社交IQa包含37,000多个QA对，用于评估模型对日常事件和情况的社会影响进行推理的能力。

Social IQa: A Social Interaction QA Benchmark for Testing Social Commonsense Intelligence. Unlike many prior benchmarks focusing on physical or taxonomic knowledge, Social IQa centers on reasoning about people's behaviors and their social impacts. For example, given an action like "Jesse attended a concert" and a question like "Why did Jesse do this?", humans can easily infer that Jesse intended to "see their favorite performers" or "enjoy the music", rather than "see what was happening inside" or "see if it worked". The action scenarios in Social IQa cover a wide range of social situations, and its answer candidates include both human-curated answers and adversarial-filtered machine-generated responses. Social IQa contains over 37,000 QA pairs to evaluate models' ability to reason about the social impacts of everyday events and situations.

搜集汇总

背景与挑战

背景概述

SIQA (Social Interaction QA) 是一个用于评估社交常识智能的问答数据集，专注于对人们日常行为及其社会影响的推理，区别于传统侧重于物理或分类学知识的基准。该数据集包含超过37,000个QA对，覆盖广泛的社交场合，答案候选者结合了人为策划和机器生成的选项，旨在测试模型在社交互动中的推理能力。