Prediction of Culture Based on Automated Detection of Multimodal Social Signals

Embodied conversational agents are required to be socially and culturally aware to bring about trust and form a<br> relationship with users. This is fundamentally important during the first few seconds of the user-agent interaction.<br> Previous research has not accounted for culture in communication style but has rather focused on the look of the<br> agent. Similarly, research has predominately focused on unimodal or bimodal channels of communication which<br> render the human-agent interaction unnatural. The first aim of this study is to investigate whether group<br> membership of high or low context cultures can be predicted based on detection of nonverbal signals in the<br> impression formation phase of a dialogue. The second aim is to investigate whether multimodal approaches would<br> lead to better and improved accuracy compared to unimodal approaches which could lead to a more natural<br> interaction. The third aim is to identify the best predictor for media skills training communication context. To do<br> this, nonverbal signals were captured during media interview training workshops. Analysis of 32 on-camera media<br> interviews revealed that a multimodal approach produces a higher accuracy at predicting high and low context<br> cultures. The findings suggest that multimodal channels of communication produce better accuracy than that of<br> the unimodal channels. The Decision Trees Classifier performed well for multimodal communication analysis.<br> These findings could contribute to the development of more natural embodied conversational agents that consider<br> different cultural communication styles.