ofekponzo/Face_Emotion_Insight_Recognition_Dataset

---
pretty_name: Multimodal Emotion & Physiological Analysis
license: mit
task_categories:
- tabular-classification
language:
- en
size_categories:
- 1K<n<10K
---

<video src="https://huggingface.co/datasets/ofekponzo/Face_Emotion_Insight_Recognition_Dataset/resolve/main/Assignment1_EDA_Presentation.mp4
" controls="controls" style="max-width: 720px;"></video>

Multimodal Emotion & Physiological Analysis

Project Overview
This project explores the relationship between physiological signals and facial micro expressions to improve emotion recognition in therapeutic settings. This research assists in determining which facial and physiological signals should be prioritized to detect clinical "misalignment" or hidden distress during therapy sessions.

1. Dataset Selection & Description
Source: The dataset is sourced from Kaggle (Face Emotion & Physiological Insight Dataset), containing pre-processed multimodal data for emotion recognition.
Size: The dataset consists of 4,998 rows and 13 features, meeting the requirement for a substantial, non-basic dataset.
Features: The dataset includes 12 numeric predictors and 1 categorical target variable:
• Physiological Measures (6 features): Electrodermal Activity (eda_mean, eda_std, eda_peaks) and Heart Rate (hr_mean, hr_std, hr_skewness).
• Facial Behavioral Measures (6 features): Intensity of Facial Action Units (face_au01, face_au06, face_au12) and Facial Landmark metrics (distances and ratios).
Research Question: How does the intensity of movement in different facial regions (Action Units) influence the prediction of emotional states, and is the lower face more reliable than the upper face for detecting negative emotions?
Target Variable: emotion_label, which classifies each data sample into one of six emotional states: Neutral, Happy, Sad, Angry, Fear, or Surprise.
Prediction Goal: To use numeric physiological and facial features to accurately classify the patient’s emotional state.

2. Data Cleaning & Preprocessing
Missing Values & Duplicates: The dataset was found to be complete with no missing values and no duplicate rows.
Fixing Typos: I applied the strip() method to the emotion label column to remove potential leading or trailing spaces. The data was kept as an 'object' type for readability during visualization.
Scaling & Normalization: I identified a significant scaling issue (e.g., heart rate max of 113.7 vs. face_au01 max of 0.42). I performed Standardization and created a copy named "df_scaled". This ensures that the algorithm evaluates the "signal" from each data channel equally.

3. Outlier Detection & Handling
Identification: Outliers were detected using Box Plots and the IQR method. Notably, there were 67 outliers in "hr_std", 51 in "face_au01", and 41 in "eda_peaks".
Handling: I decided to keep all outliers rather than removing them.
Justification: For my research question, these outliers represent the most clinically significant moments, such as peak emotional arousal or critical micro expressions. Removing them would create a "sterile" dataset incapable of detecting extreme emotional states.

4. Descriptive Statistics & Insights
Physiological Range: The mean heart rate is 81.28, with a wide range (50 to 113) indicating significant emotional variance.
Regional Intensity: The mean for a smile (face_au12) is 0.44, significantly higher than the mean for "face_au01" (0.15). This suggests that lower-face movements are numerically more "intense".
Correlations:

• Physiological Cohesion: A remarkable 0.90 correlation was found between heart rate and skin conductance, allowing the algorithm to cross-reference these metrics for reliable arousal assessment.
• Facial Synchronization: A 0.91 correlation between face_au06 and face_au12 identifies the "Duchenne Smile," helping the algorithm distinguish between social masks and genuine connection.
• Face-Body Alignme

5. Visualizations & Research Findings

Emotion Distribution:
The visualization reveals a perfectly balanced dataset with exactly 833 samples per emotion category.

Physiological Clusters:
The scatter plot reveals two distinct "data islands" (Baseline and Arousal). This indicates that the algorithm should be designed to detect physiological "jumps" rather than gradual changes.

Negative Emotion Patterns:

Upper vs. Lower Face (Research Question Answer):
While heart rate increases during 'Angry' and 'Fear' states, smile intensity (au12) drops into negative values.
This dissonance is a primary indicator of distress.

Research Questions: Analysis & Findings:

• RQ1 (Physiological Signature): The violin plot reveals a distinct physiological profile for each emotional state, allowing the algorithm to use heart rate as a reliable primary differentiator.

• RQ2 (Upper vs. Lower Face): In negative emotions, the upper face acts as an active alarm while the lower face signifies distress through its inhibition. The algorithm can accurately predict distress by identifying the gap where brows rise but smiles vanish.

• RQ3 (Structural-Physiological Link): Physiological spikes are physically anchored to structural facial changes. This allows the algorithm to use rapid geometric shifts as a reliable proxy for internal arousal jumps.

Final Conclusion

The exploratory analysis of this multimodal dataset proves that emotional recognition is most accurate when combining both physiological and behavioral signals.
By cross referencing heart rate, skin conductance, and facial action units, we can identify emotional states that might otherwise be missed by a single channel analysis.

Key Takeaways:

Reliability of Arousal: The 0.90 correlation between Heart Rate and EDA confirms that the body’s physiological response is highly cohesive, allowing the algorithm to detect stress "triggers" or sudden emotional "jumps" with high confidence.

The "Dissonance" Marker: One of the most significant findings is that negative emotions like anger and fear are best identified through regional dissonance an increase in upper-face intensity (AU01) coupled with the inhibition of lower face movement (AU12).

Clinical Significance: Retaining statistical outliers was a critical decision, as these spikes represent the most valuable clinical moments, such as breakthroughs or anxiety peaks, which are essential for the algorithm to detect in a therapeutic setting.

In conclusion, this research provides a clear "digital signature" for distress, demonstrating that while the lower face may "mask" emotions, the combined data from the upper face and internal physiology provides a transparent view of the patient's true emotional state.

搜集汇总

数据集介绍

构建方式

在情感计算与临床心理学交叉领域，Face_Emotion_Insight_Recognition_Dataset的构建体现了多模态数据融合的前沿理念。该数据集源自Kaggle平台的Face Emotion & Physiological Insight Dataset，经过系统化预处理，形成了包含4,998条样本的结构化数据。每条记录整合了12个数值型预测特征与1个分类目标变量，涵盖皮肤电活动、心率变异性等6项生理指标，以及面部动作单元强度、面部关键点几何度量等6项行为特征。数据清洗阶段确保了完整性，通过标准化处理解决了特征间量纲差异，并基于临床价值考量保留了所有统计异常值，使其能够捕捉治疗场景中极端情绪状态的关键信号。

使用方法

在应用层面，该数据集为情感识别算法开发提供了多维验证框架。研究者可基于标准化后的特征矩阵，构建融合生理信号与面部微表情的联合分类模型。使用时应充分借鉴其揭示的临床规律：将心率突变作为生理跳跃检测的时序锚点，利用面部几何特征的快速变化作为内部唤醒的代理指标。模型训练需特别注意保留异常样本的判别价值，通过捕捉上下面部动作的失调间隙与生理指标的协同跃迁，实现治疗场景中情绪错位状态的精准识别。验证阶段建议采用分层交叉验证策略，确保算法在六类情感间的泛化稳定性。

背景与挑战

背景概述

多模态情感识别作为心理学与人工智能交叉领域的前沿课题，旨在通过整合生理信号与面部微表情等多元数据，实现对人类情感状态的精准解析。Face_Emotion_Insight_Recognition_Dataset由研究团队基于Kaggle平台的开源数据构建，聚焦于临床治疗场景中的情感对齐问题。该数据集收录了涵盖六类基本情绪的4998条样本，融合了心率变异性、皮肤电活动等生理指标，以及面部动作单元强度、几何特征等行为数据，其核心研究目标在于揭示面部区域运动强度与情感状态预测之间的内在关联，特别是探索下脸部与上脸部在识别负面情绪中的可靠性差异。这一工作为开发基于多模态信号的情感计算模型提供了实证基础，推动了情感智能在心理健康辅助诊断中的应用。

当前挑战

在情感计算领域，精准识别临床环境中的隐藏情感或错位状态是一项复杂挑战，需克服单一模态数据的局限性，实现跨模态信号的协同解析。Face_Emotion_Insight_Recognition_Dataset构建过程中，研究者面临多源异构数据的对齐与标准化难题，例如生理信号与面部行为指标在量纲与分布上存在显著差异，需通过标准化处理确保算法对各通道信号的公平评估。同时，数据中的异常值处理成为关键决策点，这些峰值往往对应情感唤醒的临界时刻，保留它们虽能增强模型对极端情感的检测能力，但也可能引入噪声，影响泛化性能。此外，如何从高维特征中提取具有判别力的情感标记，如上脸部活跃与下脸部抑制之间的失调模式，仍需更精细的模型设计与验证。

常用场景

经典使用场景

在情感计算与心理健康研究领域，Face_Emotion_Insight_Recognition_Dataset 的经典使用场景集中于多模态情感识别模型的训练与验证。该数据集整合了面部微表情与生理信号，为算法提供了跨通道的情感表征数据。研究者通过分析面部动作单元强度与心率、皮肤电活动等生理指标的关联，构建能够识别六种基本情绪的分类模型。这种多模态融合方法显著提升了情感状态检测的鲁棒性，尤其在区分真实情感与社交性掩饰表情方面展现出独特价值。

解决学术问题

该数据集有效解决了情感识别研究中单模态数据局限性带来的学术挑战。通过提供同步的面部行为与生理测量数据，它支持研究者探究情感表达的多通道一致性及失调现象。具体而言，数据集助力于验证“杜兴微笑”的生理基础、揭示负面情绪中上面部与下面部活动的分离模式，并为临床情境下的情感错位检测提供量化依据。这些工作深化了情感生成机制的理论理解，推动了基于生理锚点的情感计算范式发展。

实际应用

在实际应用层面，该数据集为开发智能心理健康辅助系统提供了关键数据支撑。基于其构建的模型可集成于远程治疗平台或临床监护环境中，实时监测患者的情感状态与生理唤醒水平。系统能够通过识别面部微表情抑制与生理信号尖峰的共现模式，预警潜在的心理困扰或治疗突破时刻。此类技术有助于辅助心理医生进行客观评估，提升干预的及时性与个性化程度，在数字健康领域具有广阔的应用前景。

数据集最近研究

最新研究方向

在情感计算与心理健康交叉领域，多模态情感识别正成为研究前沿，Face_Emotion_Insight_Recognition_Dataset通过整合生理信号与面部微表情，为临床治疗中的情感对齐问题提供了新视角。当前研究聚焦于利用生理指标如心率和皮肤电活动与面部动作单元的协同分析，以揭示隐藏的情感失调模式，特别是在负性情绪识别中，上下面部区域的不一致信号被视为关键生物标记。这一方向与远程医疗和智能诊疗系统的兴起紧密相连，推动了情感人工智能在心理健康监测中的实际应用，其意义在于通过数据驱动方法提升情感识别的准确性与临床干预的时效性。