CEAP-360VR: A Continuous Physiological and Behavioral Emotion Annotation Dataset for 360 VR Videos

CEAP-360VR: A Continuous Physiological and Behavioral Emotion Annotation Dataset for 360° Videos <br> ## General Information<br> We develop the CEAP-360VR dataset to address the lack of continuously annotated behavioral and physiological datasets for 360 video VR affective computing. Accordingly, this dataset contains a) questionnaires (SSQ, IPQ, NASA-TLX); b) continuous valence-arousal annotations; c) head and eye movements as well as left and right eye pupil diameters while watching videos; d) peripheral physiological responses (ACC, EDA, SKT, BVP, HR, IBI). Our dataset also concludes the data pre-processing, data validating scripts, along with dataset description and key steps in the stage of data acquisition and pre-processing. <br> ## Dataset Structure<br> The CEAP-360VR folder contains the following six subfolders 1_Stimuli<br> 2_QuestionnaireData<br> 3_AnnotationData<br> 4_BehaviorData<br> 5_PhysioData<br> 6_Scripts<br> The following is a detailed description of each sub-file: 1_Stimuli - VideoThumbNails<br> contains the eight thumbNails for each video (.jpg)<br> - VideoInfo.json<br> contains the detailed information for eight videos 2_QuestionnaireData - PXX_Questionnaire_Data.json (X = 1, 2, ..., 32)<br> contains questionnaire data for each participant 3_AnnotationData - Raw<br> contains the raw annotation data captured from the Joy-Con joystick for each participant<br> - Transformed<br> contains the transformed valence-arousal data generated from the raw data for each participant<br> - Frame<br> contains the re-sampled annotation data from the transformed data for each participant 4_BehaviorData - Raw<br> contains the raw behavior data captured from the HTC VIVE Pro Eye Tobii Device for each participant<br> - Transformed<br> contains the transformed heam/eye movement data (pitch/yaw) generated from the raw data, as well as pupil diameter data for each participant<br> - Frame<br> contains the re-sampled behavior data generated from the transformed data for each participant<br> - HM_ScanPath<br> contains the head scanpath data generated from the transformed data for each participant<br> - EM_Fixation<br> contains the eye gaze fixation data generated from the transformed data for each participant 5_PhysioData - Raw<br> contains the raw physiological data captured from the Empatica E4 wristband for each participant<br> - Transformed<br> contains the transformed physiological data generated from the raw data for each participant<br> - Frame<br> contains the re-sampled physiological data from the transformed data for each participant 6_Scripts - Unity Project<br> contains the complete project of our user-controlled experiment (Unity 2018.4.1f1, HTC VIVE Pro Eye HMD)<br> - Data Processed<br> contains scripts that undertake the pre-processing steps for converting the raw data to the transformed/frame data in the transformed and frame folders.<br> conatins scripts for continuous annotation, behavior and physiological data analysis and visualization.<br> - CEAP-360VR_Baseline<br> contains scripts to generate processed behavioral and physiological data with V-A labels for deep learning experiments and features for machine learning experiments.<br> contains scripts to run ML and DL experiments under both subject-dependent and subject-independent model. <br> ## Dataset Description<br> The CEAP-360VR Dataset [Description.pdf](https://github.com/cwi-dis/CEAP-360VR-Dataset/blob/master/CEAP-Dataset%20Description.pdf) introduces the dataset description and key steps in the stage of data acquisition and pre-processing. <br> ## Dataset License<br> CEAP-360VR dataset is licensed under a [Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0) license]. ## Citation Please cite our paper in any published work that uses this dataset as follows:<br> - Plain Text<br> T. Xue, A. El Ali, T. Zhang, G. Ding, and P. Cesar, "CEAP-360VR: A Continuous Physiological and Behavioral Emotion Annotation Dataset for 360° Videos," in IEEE Transactions on Multimedia, doi: 10.1109/TMM.2021.3124080. - BibTex<br> @ARTICLE{Xue2021CEAP-360VR,<br> author={Xue, Tong and Ali, Abdallah El and Zhang, Tianyi and Ding, Gangyi and Cesar, Pablo},<br> journal={IEEE Transactions on Multimedia}, <br> title={CEAP-360VR: A Continuous Physiological and Behavioral Emotion Annotation Dataset for 360° Videos}, <br> year={2021},<br> volume={},<br> number={},<br> pages={1-1},<br> doi={https://doi.org/10.1109/TMM.2021.3124080}} <br> ## Usage 1. We have performed the time alignment of different types of data and <br> videos for each participant, as well as the proceesing scripts that <br> can be used to generate both the transformed and frame data. <br> Researchers can run their analysis methods on them.<br> <br> 2. For researchers who want to try other data processing methods, you can directly use the raw data. <br> ## About <br> The CEAP-360VR Dataset is maintained by Key Laboratory of Digital Performance and Simulation Technology at Beijing Institute of Technology and the Distributed &amp; Interactive Systems (DIS) research group at Centrum Wiskunde &amp; Informatica . Contact the authors<br> - Tong Xue: [e-mail], [e-mail]<br> - Abdallah El Ali: [e-mail]