Prediction of Culture Based on Automated Detection of Multimodal Social Signals

Users
require conversational virtual characters to be socially and culturally aware
in order to develop trust and relationship(s). This is dependent on the goal
and setting of the interaction. In previous research It has been suggested that
cultures that require high context (increased use of nonverbal signals) and low
context (more direct verbal communication) during communication differ in their
use of nonverbal displays. The aim of this paper is to investigate the
following: 1) whether group membership of high or low context cultures can be
predicted based on detection of nonverbal signals in the impression formation
phase of a dialogue; and 2) whether multimodal approaches would lead to better
and improved accuracy compared to unimodal approaches. As part of this study
and to collect data, nonverbal signals were captured during two media skills
training workshops. Analysis of 35 media interviews revealed that a multimodal
approach produces a higher F measure at predicting high and low context
cultures (0.772 to 0.848 vs 0.733 to 0.771 for unimodal channels). These
findings contribute to the development of conversational virtual characters
that demonstrate more culturally aware communication styles in the initial
stages of an interaction with their users.