Modality switches occur early and extend late in conceptual processing: Evidence from ERPs (Bernabeu, Willems, & Louwerse, in prep.)

Files: poster, design overview, stimuli, EEG montage used, waveforms, window 1 close-ups, subtraction topographies, critical results, fixed effects of final models, entire modeling, raw data. For best resolution, download image(s). <b>Plot waveforms within any sections of the data here</b>, and <b>access all data and code here</b>.<b>Abstract </b>We tested the relevance of modality-specific information for conceptual processing by tracking the Conceptual Modality Switch effect over time. Forty-six participants verified the relation between property words and concept words. The conceptual modalities of consecutive trials were controlled in order to produce: an Auditory-to-visual switch, a Haptic-to-visual switch, and a Visual-to-visual no-switch. Event-Related Potentials (ERPs) were time-locked to the onset of the first word in the target trials so as to measure the effect online and to avoid a task-related confound. A switch effect was found, characterized by more negative ERP amplitudes for modality switches than no-switches. It proved significant in four typical time windows from 160 to 750 milliseconds post word onset, with greater strength in posterior brain regions, and after 350 milliseconds. These results suggest that modality-specific information may be part of conceptual processing in certain tasks, but also that the early stage of processing is relatively amodal.<b></b><b></b><b></b><b></b><b></b><b></b>ReferencesCollins, J., Pecher, D., Zeelenberg, R., &amp; Coulson, S. (2011). Modality switching in a property verification task: an ERP study of what happens when candles flicker after high heels click. <i>Frontiers in Psychology, 2.</i>Hald, L. A., Marshall, J.-A., Janssen, D. P., &amp; Garnham, A. (2011). Switching modalities in a sentence verification task: ERP evidence for embodied language processing. <i>Frontiers in Psychology, 2.</i> Hauk, O. (2016). Only time will tell—Why temporal information is essential for our neuroscientific understanding of semantics. <i>Psychonomic Bulletin &amp; Review, 23</i>.<i></i>Louwerse, M., &amp; Connell, L. (2011). A taste of words: linguistic context and perceptual simulation predict the modality of words. <i>Cognitive Science, 35, </i>2, 381-98.Mahon, B.Z., &amp; Hickok, G. (2016). Arguments about the nature of concepts: Symbols, embodiment, and beyond. <em>Psychonomic Bulletin &amp; Review, 23, </em>941-958.