Audiovisual Integration Varies with Target and Environment Richness in Immersive Virtual Reality: Supplementary Material

We are continually
bombarded by information arriving to each of our senses; however, the brain seems
to effortlessly integrate this separate information into a unified percept.
Although multisensory integration has been researched extensively using simple
computer tasks and stimuli, much less is known about how multisensory
integration functions in real-world contexts. Additionally, several recent studies have
demonstrated that multisensory integration varies tremendously across
naturalistic stimuli. Virtual reality can be used to study multisensory
integration in realistic settings because it combines realism with precise
control over the environment and stimulus presentation. In the current study,
we investigated whether multisensory integration as measured by the redundant
signals effects (RSE) is observable in naturalistic environments using virtual
reality and whether it differs as a function of target and/or environment cue-richness. Participants detected auditory, visual, and
audiovisual targets which varied in cue-richness within three distinct virtual
worlds that also varied in cue-richness. We demonstrated integrative effects in
each environment-by-target pairing and further showed a modest effect on multisensory
integration as a function of target cue-richness but only in the cue-rich
environment. Our study is the first to definitively show that minimal and more
naturalistic tasks elicit comparable redundant signals effects. Our results also
suggest that multisensory integration may function differently depending on the
features of the environment. The results
of this study have important implications in the design of virtual multisensory
environments that are currently being used for training, educational, and
entertainment purposes.

我们不断受到各种感官信息的冲击；然而，大脑似乎能够毫不费力地将这些分离的信息整合为统一的感觉。尽管多感官整合在利用简单的计算机任务和刺激的研究中得到了广泛的探讨，但在现实世界情境中多感官整合的功能了解甚少。此外，一些近期研究已证明，在自然刺激中多感官整合存在巨大的差异。虚拟现实技术能够用于研究现实环境中的多感官整合，因为它将真实性与对环境和刺激呈现的精确控制相结合。在本研究中，我们探讨了使用虚拟现实技术测量多感官整合的冗余信号效应（RSE）是否可在自然环境中观察到，以及它是否随目标及/或环境线索的丰富性而有所不同。参与者检测了不同线索丰富性的听觉、视觉和视听目标，这些目标分布在三个不同的虚拟世界中，这些虚拟世界也具有不同的线索丰富性。我们在每种环境和目标的配对中观察到了整合效应，并进一步展示了线索丰富性对多感官整合的微小影响，但这种影响仅在线索丰富的环境中出现。我们的研究首次明确地显示出，最小化和更自然化的任务能够引发相似的重叠信号效应。我们的研究结果还表明，多感官整合可能根据环境的特征而表现出不同的功能。本研究的结果对于设计目前用于训练、教育和娱乐目的的虚拟多感官环境具有重要的启示。