Do Bodily Expressions Compete with Facial Expressions? Time Course of Integration of Emotional Signals from the Face and the Body

The decoding of social signals from nonverbal cues plays a vital role in the social interactions of socially gregarious animals such as humans. Because nonverbal emotional signals from the face and body are normally seen together, it is important to investigate the mechanism underlying the integration of emotional signals from these two sources. We conducted a study in which the time course of the integration of facial and bodily expressions was examined via analysis of event-related potentials (ERPs) while the focus of attention was manipulated. Distinctive integrating features were found during multiple stages of processing. In the first stage, threatening information from the body was extracted automatically and rapidly, as evidenced by enhanced P1 amplitudes when the subjects viewed compound face-body images with fearful bodies compared with happy bodies. In the second stage, incongruency between emotional information from the face and the body was detected and captured by N2. Incongruent compound images elicited larger N2s than did congruent compound images. The focus of attention modulated the third stage of integration. When the subjects' attention was focused on the face, images with congruent emotional signals elicited larger P3s than did images with incongruent signals, suggesting more sustained attention and elaboration of congruent emotional information extracted from the face and body. On the other hand, when the subjects' attention was focused on the body, images with fearful bodies elicited larger P3s than did images with happy bodies, indicating more sustained attention and elaboration of threatening information from the body during evaluative processes.

从非言语线索中解析社会信号，对于人类这类社会性群居动物的社交互动至关重要。由于面部与躯体传递的非言语情绪信号通常同时出现，因此探究这两类来源的情绪信号整合机制具有重要学术价值。本研究通过事件相关电位（event-related potentials, ERPs）分析，并操纵注意焦点，考察了面部与躯体表情整合的时间进程。研究在信息加工的多个阶段均发现了独特的整合特征：在第一阶段，躯体传递的威胁性信息可被自动快速提取，具体表现为相较于呈现愉悦躯体的复合面部-躯体图像，被试观看带有恐惧躯体的复合图像时，其P1波幅显著升高；在第二阶段，面部与躯体情绪信息间的不一致性可被检测到，该效应由N2成分体现，不一致的复合图像诱发的N2波幅显著大于一致的复合图像；注意焦点会调控整合过程的第三阶段：当被试将注意聚焦于面部时，情绪信号一致的图像较不一致图像可诱发更大的P3波幅，表明被试对从面部与躯体提取的一致情绪信息会投入更持久的注意并进行精细化加工；而当被试将注意聚焦于躯体时，带有恐惧躯体的图像较愉悦躯体图像可诱发更大的P3波幅，表明在评估过程中，被试会对躯体传递的威胁性信息投入更持久的注意并进行精细化加工。