Emotion Recognition for Affective human digital twin by means of virtual reality enabling technologies

We introduce a new bimodal dataset recorded during affect elicitation by means of audio-visual stimuli for human emotion recognition based on facial and corporal expressions. Our dataset was collected using three devices: an RGB camera, Kinect 1, and Kinect 2. The Kinect 1 and Kinect 2 sensors provide 121 and 1347 face key points, respectively, offering a more comprehensive analysis of facial expressions. Additionally, for the 2D RGB sequences, we utilized the feature points provided by the open-source OpenFace, which includes 2D 68 facial landmarks. From these landmarks, we selected 26 facial points that were most relevant for our emotion recognition task.
To gather the data, we conducted experiments involving 17 participants. We captured both facial and skeleton keypoints, allowing for a comprehensive understanding of the participants' emotional expressions. By combining the RGB and RGB-D data from the various devices, our dataset provides a rich and diverse set of information for human emotion recognition research.

This new dataset not only expands the available resources for studying human emotions but also offers a more detailed analysis with the increased number of facial keypoints provided by the Kinect sensors. Researchers can leverage this dataset to develop and evaluate more accurate and robust models for human emotion recognition, ultimately advancing our understanding of how emotions are expressed through facial and corporal cues.
Please cite as:
K. Amara, O. Kerdjidj and N. Ramzan, "Emotion Recognition for Affective human digital twin by means of virtual reality enabling technologies," in IEEE Access, doi: 10.1109/ACCESS.2023.3285398.

Please state your name, contact details (e-mail), institution, and position, as well as the reason for requesting access to our database.

For additional info contact:

[e-mail] or [e-mail]

[e-mail]

[e-mail]