A Multimodal Dataset for Mixed Emotion Recognition

ABSTRACT: Mixed emotions have attracted increasing interest recently, but existing datasets rarely focus on mixed emotion recognition from multimodal signals, hinder the affective computing of mixed emotions. On this basis, we present a multimodal dataset with four kinds of signals recorded during watching mixed and non-mixed emotion videos. To ensure effective emotion induction, we first implemented a rule-based video filtering step to select the videos that could elicit stronger positive, negative, and mixed emotions. Then, an experiment with 80 participants was conducted, in which the data of EEG, GSR, PPG and frontal face videos were recorded while they watching the selected video clips. We also recorded the subjective emotional rating on PANAS, VAD, and amusement-disgust dimensions. In total, the dataset consists of multimodal signal data and self-assessment data from 73 participants (7 of the 80 participants’ data were discarded due to the deficiency in physiological signal). We also present technical validations for emotion induction and mixed emotion classification from physiological signals and face videos.