K-EmoCon, a multimodal sensor dataset for continuous emotion recognition in naturalistic conversations

ABSTRACT: Recognizing emotions during social interactions has many potential applications with the popularization of low-cost mobile sensors, but a challenge remains with the lack of naturalistic affective interaction data. Most existing emotion datasets do not support studying idiosyncratic emotions arising in the wild as they were collected in constrained environments. Therefore, studying emotions in the context of social interactions requires a novel dataset, and K-EmoCon is such a multimodal dataset with comprehensive annotations of continuous emotions during naturalistic conversations. The dataset contains multimodal measurements, including audiovisual recordings, EEG, and peripheral physiological signals, acquired with off-the-shelf devices from 16 sessions of approximately 10-minute long paired debates on a social issue. Distinct from previous datasets, it includes emotion annotations from all three available perspectives: self, debate partner, and external observers. Raters annotated emotional displays at intervals of every 5 seconds while viewing the debate footage, in terms of arousal-valence and 18 additional categorical emotions. The resulting K-EmoCon is the first publicly available emotion dataset accommodating the multiperspective assessment of emotions during social interactions. +---------------------------------------+ | Changelog (last updated: Jul 7, 2020) | +---------------------------------------+ * Version 1.0.0 (Jul 7, 2020): - Updated emotion_annotations.tar.gz: - Updated aggregated external annotations to support the reproduction of technical validation results. * Version 0.2.0 (May 11, 2020): - Added data_quality_tables.tar.gz - Updated emotion_annotations.tar.gz - Newly added aggregated external annotations. - Updated metadata.tar.gz: - Added a new column to data_availability.csv to show the availability of aggregated external annotations. * Version 0.1.0 (Apr 25, 2020): - Added debate_audios.tar.gz - Added debate_recordings.tar.gz - Added e4_data.tar.gz - Added emotion_annotations.tar.gz (self, partner, external) - Added metadata.tar.gz - Added neurosky_polar_data.tar.gz摘要：随着低成本移动传感器的普及，社交互动中的情绪识别拥有诸多潜在应用，但目前仍面临缺乏自然主义情感交互数据的挑战。现有多数情绪数据集均在受限环境中采集，无法支持对野外场景下产生的个性化情绪展开研究。因此，开展社交互动场景下的情绪研究亟需全新的数据集，而K-EmoCon正是这样一个针对自然对话场景中连续情绪进行全面标注的多模态数据集。该数据集包含多模态采集数据，涵盖视听录制、脑电图（EEG）以及外周生理信号，采用商用现成设备采集自16组时长约10分钟的配对辩论场景，辩论主题均为社会议题。与此前数据集不同的是，本数据集包含来自三类可用视角的情绪标注：自我视角、辩论搭档视角以及外部观察者视角。标注人员在观看辩论录像时，以每5秒为间隔进行情绪标注，标注维度包括唤醒度-效价（arousal-valence）以及18种额外的分类情绪。最终发布的K-EmoCon是首个可支持社交互动场景下多视角情绪评估的公开可用情绪数据集。 更新日志（最后更新时间：2020年7月7日） * 版本1.0.0（2020年7月7日）： - 更新emotion_annotations.tar.gz： - 更新聚合外部标注数据，以支持技术验证结果的复现。 * 版本0.2.0（2020年5月11日）： - 新增data_quality_tables.tar.gz - 更新emotion_annotations.tar.gz： - 新增聚合外部标注数据。 - 更新metadata.tar.gz： - 在data_availability.csv中新增一列，用于展示聚合外部标注数据的可用性。 * 版本0.1.0（2020年4月25日）： - 新增debate_audios.tar.gz - 新增debate_recordings.tar.gz - 新增e4_data.tar.gz - 新增emotion_annotations.tar.gz（包含自我、搭档、外部视角标注） - 新增metadata.tar.gz - 新增neurosky_polar_data.tar.gz