Visual intensity-dependent response latencies predict perceived audio-visual simultaneity - Data

Simultaneity judgement (SJ) and temporal order judgement (TOJ) tasks are commonly used to investigate the temporal processes involved in multisensory perception. From these tasks, it is possible to estimate a point of subjective simultaneity (PSS), which refers to the audio-visual offset where simultaneity is perceived. Despite the tasks usually involving identical stimuli, the PSSs have been shown to be uncorrelated. The multisensory correlation detector (MCD) model (Parise & Ernst, 2016) predicts this lack of correlation, by proposing that different decisional weightings are applied to common, early audio-visual processing mechanisms. In the included files, we evaluate an additional component to the MCD model: an intensity-dependent processing delay that was estimated from simple reaction times. The Matlab codes titled 'M1' to 'M4' will load .mat files that contain behavioural data and run the various models which we compared. Our behavioural data was obtained using the SJ (titled p1_sj_intensGenfile to p20_sj_intensGenfile) and TOJ tasks (titled p1_tj_intensGenfile to p20_tj_intensGenfile), measured at four visual intensities, with a fixed auditory intensity. Each file contains 4 matrices, titled 'dataIntensity1' to 'dataIntensity4', with 1 being the lowest visual intensity. Each matrix contains 2 columns: column 1 represents the stimulus onset asynchrony (with negative values indicating an audio-leading stimulus), whereas column 2 represents the relative frequency of 'simultaneous' and 'light leading' responses in the simultaneity judgement and temporal order judgement tasks, respectively. ‘pssSJ’ and ‘pssTOJ’ contain the PSSs estimated for each observer using atheoretical fits. The results show that the added early processing delay predicts the intensity-dependence of perceived simultaneity, without the need for additional decisional factors. References: Parise, C. V., & Ernst, M. O. (2016). Correlation detection as a general mechanism for multisensory integration. Nature Communications, 7. https://doi.org/10.1038/ncomms11543

同时性判断（Simultaneity Judgement, SJ）与时序判断（Temporal Order Judgement, TOJ）任务是探究多感官知觉涉及的时序加工过程的常用范式。通过此类任务，可以估算出主观同时性点（Point of Subjective Simultaneity, PSS）——即个体感知到刺激同时呈现时的视听偏移量。尽管两类任务所使用的刺激完全一致，但已有研究表明二者测得的PSS并无相关性。多感官相关检测器（Multisensory Correlation Detector, MCD）模型（Parise与Ernst, 2016）通过提出对通用的早期视听加工机制施加不同决策权重的假设，对这一无相关性现象做出了预测。

本研究附带的文件中，我们对MCD模型的一项额外组成部分进行了评估：即基于简单反应时估算得到的强度依赖加工延迟。名为'M1'至'M4'的Matlab代码可加载包含行为数据的.mat格式文件，并运行我们所比对的各类模型。我们的行为数据通过SJ任务（对应文件名为p1_sj_intensGenfile至p20_sj_intensGenfile）与TOJ任务（对应文件名为p1_tj_intensGenfile至p20_tj_intensGenfile）获取，实验中设置了4种视觉强度，且听觉强度固定不变。每个文件均包含4个名为dataIntensity1至dataIntensity4的矩阵，其中1代表最低视觉强度。每个矩阵包含2列数据：第1列为刺激呈现异步性（Stimulus Onset Asynchrony, SOA，负值表示听觉先导刺激），第2列分别对应同时性判断任务中“同时”反应的相对频率，与时序判断任务中“视觉先导”反应的相对频率。文件中的pssSJ与pssTOJ分别存储了通过非理论拟合为每位被试估算得到的SJ与TOJ任务PSS。

研究结果表明，新增的早期加工延迟可解释知觉同时性的强度依赖性，且无需额外引入决策因素。

参考文献：Parise, C. V., & Ernst, M. O. (2016). Correlation detection as a general mechanism for multisensory integration. Nature Communications, 7. https://doi.org/10.1038/ncomms11543