Modality switches occur early and extend late in conceptual processing: Evidence from ERPs (Bernabeu, Willems, & Louwerse, in prep.)

Files: poster, design overview, stimuli, EEG montage used, waveforms, difference topographies, critical statistics, fixed effects of final models, entire modeling, raw data. For best resolution, please download image(s). Also see <b>waveforms interactively within sections of the data</b>, and finally <b>the entire data set.</b><b>Abstract </b>This experiment investigates the relevance of perceptual simulation in conceptual processing by revisiting the conceptual modality switch paradigm. In each trial, participants verified the relation between a property word and a concept word. These linguistic stimuli were modality-normed, and covertly induced switches in conceptual modality across some trials—specifically, auditory-to-visual and haptic-to-visual switches. Those switches were compared to the no-switch condition—visual-to-visual. ERPs were time-locked to the first word in the target trials so as to reduce within-trial confounds and measure the conceptual modality switch on-line. A widespread effect was found, characterized by more negative ERP amplitudes for modality-switching than not switching. This effect proved significant in four typical time windows from 160 to 750 ms post word onset, with particular strength in the N400 window and in Posterior brain regions. The results suggest that perceptual simulation functionally contributes to conceptual processing, but also that earlier processing is relatively symbolic.<b></b><b></b><b></b><b></b><b></b><b></b>ReferencesCollins, J., Pecher, D., Zeelenberg, R., &amp; Coulson, S. (2011). Modality switching in a property verification task: an ERP study of what happens when candles flicker after high heels click. <i>Frontiers in Psychology, 2.</i>Hald, L. A., Marshall, J.-A., Janssen, D. P., &amp; Garnham, A. (2011). Switching modalities in a sentence verification task: ERP evidence for embodied language processing. <i>Frontiers in Psychology, 2.</i> Hauk, O. (2016). Only time will tell—Why temporal information is essential for our neuroscientific understanding of semantics. <i>Psychonomic Bulletin &amp; Review, 23</i>.<i></i>Louwerse, M., &amp; Connell, L. (2011). A taste of words: linguistic context and perceptual simulation predict the modality of words. <i>Cognitive Science, 35, </i>2, 381-98.Mahon, B.Z., &amp; Hickok, G. (2016). Arguments about the nature of concepts: Symbols, embodiment, and beyond. <em>Psychonomic Bulletin &amp; Review, 23, </em>941-958.