Experiment data.|面部表情识别数据集|文化差异数据集

Facial expression recognition (FER) is significantly influenced by the cultural background (CB) of observers and the masking conditions of the target face. This study aimed to clarify these factors’ impact on FER, particularly in machine-learning datasets, increasingly used in human-computer interaction and automated systems. We conducted an FER experiment with East Asian participants and compared the results with the FERPlus dataset, evaluated by Western raters. Our novel analysis approach focused on variability between images and participants within a "majority" category and the eye-opening rate of target faces, providing a deeper understanding of FER processes. Notable findings were differences in "fear" perception between East Asians and Westerners, with East Asians more likely to interpret "fear" as "surprise." Masking conditions significantly affected emotion categorization, with "fear" perceived by East Asians for non-masked faces interpreted as "surprise" for masked faces. Then, the emotion labels were perceived as different emotions across categories in the masking condition, rather than simply lower recognition rates or confusion as in existing studies. Additionally, "sadness" perceived by Westerners was often interpreted as "disgust" by East Asians. These results suggest that one-to-one network learning models, commonly trained using majority labels, might overlook important minority response information, potentially leading to biases in automated FER systems. In conclusion, FER dataset characteristics differ depending on the target face’s masking condition and the diversity among evaluation groups. This study highlights the need to consider these factors in machine-learning-based FER that relies on human-judged labels, to contribute to the development of more nuanced and fair automated FER systems. Our findings emphasize the novelty of our approach compared to existing studies and the importance of incorporating a broader range of human variability in FER research, setting the stage for future evaluations of machine learning classifiers on similar data.

面部表情识别（FER）显著受观察者的文化背景（CB）及目标面部遮蔽条件的影响。本研究旨在阐明这些因素对FER的影响，特别是在日益应用于人机交互和自动化系统的机器学习数据集中。我们针对东亚参与者进行了FER实验，并将结果与由西方评审员评估的FERPlus数据集进行了比较。我们新颖的分析方法集中于“多数”类别内图像和参与者之间的差异性以及目标面部引人注目的率，从而更深入地理解FER过程。显著的发现是东亚人与西方人在“恐惧”感知上的差异，东亚人更可能将“恐惧”解释为“惊讶”。遮蔽条件显著影响了情绪分类，东亚人对非遮蔽面部的“恐惧”感知在遮蔽条件下被解释为“惊讶”。然后，在遮蔽条件下，情绪标签被感知为不同类别中的不同情绪，而不仅仅是现有研究中的识别率降低或混淆。此外，西方人感知到的“悲伤”常被东亚人解释为“厌恶”。这些结果表明，通常使用多数标签训练的一对一网络学习模型可能会忽视重要的少数响应信息，可能导致自动化FER系统中出现偏差。总之，FER数据集的特征取决于目标面部的遮蔽条件和评估群体之间的多样性。本研究强调了在依赖人工判断标签的机器学习FER中考虑这些因素的必要性，以促进更细腻和公平的自动化FER系统的发展。我们的发现强调了与现有研究相比我们方法的新颖性以及将更广泛的人类差异性纳入FER研究的重要性，为未来在类似数据上评估机器学习分类器奠定了基础。