Prediction of Culture Based on Automated Detection of Multimodal Social Signals

Embodied conversational agents are required to be socially and culturally aware to bring about trust and form a relationship with users. This is fundamentally important during the first few seconds of the user-agent interaction. Previous research has not accounted for culture in communication style but has rather focused on the look of the agent. Similarly, research has predominately focused on unimodal or bimodal channels of communication which render the human-agent interaction unnatural. The first aim of this study is to investigate whether group membership of high or low context cultures can be predicted based on detection of nonverbal signals in the impression formation phase of a dialogue. The second aim is to investigate whether multimodal approaches would lead to better and improved accuracy compared to unimodal approaches which could lead to a more natural interaction. The third aim is to identify the best predictor for media skills training communication context. To do this, nonverbal signals were captured during media interview training workshops. Analysis of 32 on-camera media interviews revealed that a multimodal approach produces a higher accuracy at predicting high and low context cultures. The findings suggest that multimodal channels of communication produce better accuracy than that of the unimodal channels. The Decision Trees Classifier performed well for multimodal communication analysis. These findings could contribute to the development of more natural embodied conversational agents that consider different cultural communication styles.