Data from: Neural correlates of multisensory enhancement in audiovisual narrative speech perception: a fMRI investigation

This fMRI study investigated the effect of seeing articulatory movements
of a speaker while listening to a naturalistic narrative stimulus. It had
the goal to identify regions of the language network showing multisensory
enhancement under synchronous audiovisual conditions. We expected this
enhancement to emerge in regions known to underlie the integration of
auditory and visual information such as the posterior superior temporal
gyrus as well as parts of the broader language network, including the
semantic system. To this end we presented 53 participants with a
continuous narration of a story in auditory alone, visual alone, and both
synchronous and asynchronous audiovisual speech conditions while recording
brain activity using BOLD fMRI. We found multisensory enhancement in an
extensive network of regions underlying multisensory integration and parts
of the semantic network as well as extralinguistic regions not usually
associated with multisensory integration, namely the primary visual cortex
and the bilateral amygdala. Analysis also revealed involvement of thalamic
brain regions along the visual and auditory pathways more commonly
associated with early sensory processing. We conclude that under natural
listening conditions, multisensory enhancement not only involves sites of
multisensory integration but many regions of the wider semantic network
and includes regions associated with extralinguistic sensory, perceptual
and cognitive processing.