KU-EMOTION

The dataset was collected to advance research on multimodal emotion datasets that combine EEG and facial video data using consumer-grade devices in naturalistic settings. This dataset aims to ensure ecological validity by capturing authentic emotional responses in comfortable environments. It contains emotional expressions when two contrasting stimuli were presented to the participants, reflected in both brain activity and facial movements. The participants were instructed to focus on the laptop screen while the stimuli were presented in a fully automated sequence. Each session consisted of a ten-second countdown, followed by the first video, a brief black screen, and then the second video, with the program closing automatically at the end. Since the dataset includes EEG and facial videos recorded simultaneously, it can be used for both unimodal and cross-modal research on emotional experiences.

At the root level, the dataset is organized into three main folders which are raw folder, processed folder, stimuli folder.

The raw data folder contains subfolders numbered 01 to 67, with each number representing a unique participant. Within every participant folder, the dataset stores several files, such as EEG_data_{timestamp}.json, which contains the recorded EEG signals, video_data_{timestamp}.avi, which stores the raw video recordings, and video_temporal_info_{timestamp}.json, which provides precise timestamp information for each frame in the corresponding video file. Additionally, subject_information.json contains participant-specific metadata, including sex, age, and unique identifier. The file stimuli_temporal_info_{timestamp}.json logs the onset times of the countdown sequence, video stimuli, and the inter-stimulus black screen interval. At the root of the raw data directory, video_stimuli_data_sequence.json documents the sequence of stimuli presented to all participants.

The processed folder stores all files produced after data preprocessing. The processed_data.pickle file holds the formatted dataset prepared for analysis, while split_information.pickle contains the details of how the data can be partitioned for training, validation, and testing of models. The stimuli folder includes the two video files, 1.mp4 and 2.mp4, used as stimuli during the experiment.

The code for converting raw data into processed data and for reading the processed data is available at this link: https://github.com/mj-raihan/KU-EMOTION

The software used for collecting sensor data and presenting stimuli can be accessed here: https://github.com/mj-raihan/NeuroCue