allenai/social_i_qa

---
language:
- en
paperswithcode_id: social-iqa
pretty_name: Social Interaction QA
dataset_info:
features:
- name: context
dtype: string
- name: question
dtype: string
- name: answerA
dtype: string
- name: answerB
dtype: string
- name: answerC
dtype: string
- name: label
dtype: string
splits:
- name: train
num_bytes: 6389954
num_examples: 33410
- name: validation
num_bytes: 376508
num_examples: 1954
download_size: 2198056
dataset_size: 6766462
---

# Dataset Card for "social_i_qa"

## Table of Contents
- [Dataset Description](#dataset-description)
- [Dataset Summary](#dataset-summary)
- [Supported Tasks and Leaderboards](#supported-tasks-and-leaderboards)
- [Languages](#languages)
- [Dataset Structure](#dataset-structure)
- [Data Instances](#data-instances)
- [Data Fields](#data-fields)
- [Data Splits](#data-splits)
- [Dataset Creation](#dataset-creation)
- [Curation Rationale](#curation-rationale)
- [Source Data](#source-data)
- [Annotations](#annotations)
- [Personal and Sensitive Information](#personal-and-sensitive-information)
- [Considerations for Using the Data](#considerations-for-using-the-data)
- [Social Impact of Dataset](#social-impact-of-dataset)
- [Discussion of Biases](#discussion-of-biases)
- [Other Known Limitations](#other-known-limitations)
- [Additional Information](#additional-information)
- [Dataset Curators](#dataset-curators)
- [Licensing Information](#licensing-information)
- [Citation Information](#citation-information)
- [Contributions](#contributions)

## Dataset Description

- **Homepage:** [https://leaderboard.allenai.org/socialiqa/submissions/get-started](https://leaderboard.allenai.org/socialiqa/submissions/get-started)
- **Repository:** [More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
- **Paper:** [More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
- **Point of Contact:** [More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
- **Size of downloaded dataset files:** 2.20 MB
- **Size of the generated dataset:** 6.76 MB
- **Total amount of disk used:** 8.97 MB

### Dataset Summary

We introduce Social IQa: Social Interaction QA, a new question-answering benchmark for testing social commonsense intelligence. Contrary to many prior benchmarks that focus on physical or taxonomic knowledge, Social IQa focuses on reasoning about people’s actions and their social implications. For example, given an action like "Jesse saw a concert" and a question like "Why did Jesse do this?", humans can easily infer that Jesse wanted "to see their favorite performer" or "to enjoy the music", and not "to see what's happening inside" or "to see if it works". The actions in Social IQa span a wide variety of social situations, and answer candidates contain both human-curated answers and adversarially-filtered machine-generated candidates. Social IQa contains over 37,000 QA pairs for evaluating models’ abilities to reason about the social implications of everyday events and situations. (Less)

### Supported Tasks and Leaderboards

[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)

### Languages

[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)

## Dataset Structure

### Data Instances

#### default

- **Size of downloaded dataset files:** 2.20 MB
- **Size of the generated dataset:** 6.76 MB
- **Total amount of disk used:** 8.97 MB

An example of 'validation' looks as follows.
```
{
"answerA": "sympathetic",
"answerB": "like a person who was unable to help",
"answerC": "incredulous",
"context": "Sydney walked past a homeless woman asking for change but did not have any money they could give to her. Sydney felt bad afterwards.",
"label": "1",
"question": "How would you describe Sydney?"
}
```

### Data Fields

The data fields are the same among all splits.

#### default
- `context`: a `string` feature.
- `question`: a `string` feature.
- `answerA`: a `string` feature.
- `answerB`: a `string` feature.
- `answerC`: a `string` feature.
- `label`: a `string` feature.

### Data Splits

| name |train|validation|
|-------|----:|---------:|
|default|33410| 1954|

## Dataset Creation

### Curation Rationale

[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)

### Source Data

#### Initial Data Collection and Normalization

[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)

#### Who are the source language producers?

[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)

### Annotations

#### Annotation process

[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)

#### Who are the annotators?

[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)

### Personal and Sensitive Information

[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)

## Considerations for Using the Data

### Social Impact of Dataset

[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)

### Discussion of Biases

[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)

### Other Known Limitations

[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)

## Additional Information

### Dataset Curators

[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)

### Licensing Information

[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)

### Citation Information

```

```

### Contributions

Thanks to [@bhavitvyamalik](https://github.com/bhavitvyamalik), [@thomwolf](https://github.com/thomwolf), [@patrickvonplaten](https://github.com/patrickvonplaten), [@lewtun](https://github.com/lewtun) for adding this dataset.

原始信息汇总

数据集概述

数据集名称

• 名称: Social Interaction QA
• 简称: Social IQa

数据集信息

• 特征:
  • context: 字符串类型
  • question: 字符串类型
  • answerA: 字符串类型
  • answerB: 字符串类型
  • answerC: 字符串类型
  • label: 字符串类型
• 数据分割:
  • train: 33410个样本，6389954字节
  • validation: 1954个样本，376508字节
• 下载大小: 2198056字节
• 数据集大小: 6766462字节

数据集描述

• 概述: Social IQa是一个专注于测试社会常识智能的问题回答基准。与许多关注物理或分类知识的先前基准不同，Social IQa侧重于推理人们的行动及其社会影响。该数据集包含超过37,000个QA对，用于评估模型对日常事件和情况的社会影响的推理能力。

数据集结构

• 数据实例: 示例包括context, question, answerA, answerB, answerC, 和 label。
• 数据字段:
  • context: 描述情境的字符串
  • question: 提出的问题
  • answerA, answerB, answerC: 可能的答案
  • label: 正确答案的标签
• 数据分割:
  • train: 33410个样本
  • validation: 1954个样本

贡献者

• @bhavitvyamalik
• @thomwolf
• @patrickvonplaten
• @lewtun

搜集汇总

数据集介绍

构建方式

在社交智能问答领域，Social IQa数据集的构建体现了对社交常识推理的深度探索。该数据集通过精心设计的情境描述与问题对，结合人类专家标注与对抗性机器生成候选答案的筛选机制，形成了超过37,000个问答对。构建过程聚焦于日常社交互动中的行为动机与情感反应，确保了数据在多样社会场景下的覆盖广度与逻辑深度，为模型提供了丰富的社交推理训练素材。

特点

Social IQa数据集的核心特点在于其专注于社交互动的隐含逻辑推理，而非传统的物理或分类知识。每个数据实例包含一个情境描述、一个开放式问题以及三个候选答案，其中正确答案由人类标注，干扰项则通过对抗性生成技术优化，以增强模型的判别能力。数据集涵盖广泛的社会情境，如情感反应、行为动机等，旨在挑战模型对复杂社交动态的理解与推断能力。

使用方法

该数据集适用于评估和训练自然语言处理模型在社交常识推理方面的性能。用户可通过加载数据集的标准分割（训练集与验证集），利用情境、问题及候选答案字段构建多选问答任务。典型应用包括模型微调、基准测试以及社交智能研究，通过预测标签字段对应的正确答案，量化模型对社交隐含意义的捕捉能力，推动人工智能在人性化交互领域的进展。

背景与挑战

背景概述

在人工智能领域，社会常识推理是衡量机器智能深度的重要维度。由艾伦人工智能研究所于2019年发布的Social IQa数据集，旨在填补社会交互理解方面的研究空白。该数据集由Maarten Sap等学者主导构建，核心研究问题聚焦于模型对日常社会情境中人类行为及其隐含动机的推理能力。通过提供超过37,000个问答对，Social IQa推动了自然语言处理模型从表层语义理解向深层社会认知的跨越，对情感计算、对话系统及可解释人工智能等领域产生了深远影响。

当前挑战

Social IQa数据集所针对的社会常识推理任务，其挑战在于模型需超越字面意义，捕捉复杂情境中微妙的情感、意图与道德判断。这要求模型具备跨语境的文化敏感性和心理理论能力，而现有模型往往在隐含逻辑与多义性理解上表现不足。在构建过程中，数据收集面临社会情境多样性与标注一致性的平衡难题，需通过人工筛选与对抗性生成相结合的方式确保答案的合理性与挑战性，同时避免引入文化偏见与主观歧义。

常用场景

经典使用场景

在自然语言处理领域，社会常识推理是评估模型智能水平的关键维度。Social IQa数据集通过提供超过37,000个社会互动情境的问答对，成为测试模型理解人类行为动机及社会影响的经典基准。该数据集要求模型基于给定情境，从多个候选答案中选出最符合社会常识的选项，例如推断人物情感反应或行为原因，从而衡量模型对社会隐含规则的掌握程度。

解决学术问题

该数据集主要解决了人工智能领域对社会常识形式化建模的难题。传统问答系统往往局限于事实性知识，而Social IQa将研究焦点转向社会情境中的意图、情感和道德推理。它填补了机器理解人类社交行为背后逻辑的空白，推动了常识推理从物理世界到社会领域的拓展，为构建更具人文关怀的智能体提供了理论基础与评估标准。

衍生相关工作

围绕Social IQa数据集，学术界衍生出多项经典研究。例如，基于Transformer的模型如BERT和RoBERTa被广泛用于该任务的基线测试，推动了预训练语言模型在社会推理方向的微调策略创新。同时，该数据集启发了后续社会常识基准的构建，如PIQA和CommonsenseQA的拓展工作，形成了社会智能评估的研究脉络。