Detection and Recognition of Asynchronous Auditory/Visual Speech: Effects of Age, Hearing Loss, and Talker Accent

This investigation examined age-related differences in auditory-visual (AV) integration as reflected on perceptual judgments of temporally misaligned AV English sentences spoken by native English and native Spanish talkers. In the detection task, it was expected that slowed auditory temporal processing of older participants, relative to younger participants, would be manifest as a shift in the range over which participants would judge asynchronous stimuli as synchronous (referred to as the “AV simultaneity window”). The older participants were also expected to exhibit greater declines in speech recognition for asynchronous AV stimuli than younger participants. Talker accent was hypothesized to influence listener performance, with older listeners exhibiting a greater narrowing of the AV simultaneity window and much poorer recognition of asynchronous AV foreign-accented speech compared to younger listeners. Participant groups included younger and older participants with normal hearing and older participants with hearing loss. Stimuli were video recordings of sentences produced by native English and native Spanish talkers. The video recordings were altered in 50 ms steps by delaying either the audio or video onset. Participants performed a detection task in which the judged whether the sentences were synchronous or asynchronous, and performed a recognition task for multiple synchronous and asynchronous conditions. Both the detection and recognition tasks were conducted at the individualized signal-to-noise ratio (SNR) corresponding to approximately 70% correct speech recognition performance for synchronous AV sentences. Older listeners with and without hearing loss generally showed wider AV simultaneity windows than younger listeners, possibly reflecting slowed auditory temporal processing in auditory lead conditions and reduced sensitivity to asynchrony in auditory lag conditions. However, older and younger listeners were affected similarly by misalignment of auditory and visual signal onsets on the speech recognition task. This suggests that older listeners are negatively impacted by temporal misalignments for speech recognition, even when they do not notice that the stimuli are asynchronous. Overall, the findings show that when listener performance is equated for simultaneous AV speech signals, age effects are apparent in detection judgments but not in recognition of asynchronous speech.

本研究旨在探讨年龄差异对听觉-视觉（AV）整合的影响，这种影响体现在本族语英语和西班牙语说话者对时间错位的AV英语句子的感知判断上。在检测任务中，预计与年轻参与者相比，老年参与者的听觉时间处理变慢将表现为参与者判断异步刺激为同步刺激的范围发生偏移（称为“AV同时性窗口”）。此外，预计老年参与者在识别异步AV刺激的语音方面的下降将大于年轻参与者。研究者假设说话者的口音会影响听者的表现，老年听者的AV同时性窗口将更为狭窄，对异步AV口音语音的识别能力将远逊于年轻听者。参与者群体包括听力正常的年轻和老年参与者，以及有听力损失的老年参与者。刺激材料为母语为英语和西班牙语的说话者产生的句子视频记录。视频记录通过延迟音频或视频的开始时间以50毫秒的步长进行修改。参与者执行了一个检测任务，判断句子是同步还是异步，并在多个同步和异步条件下执行了一个识别任务。检测和识别任务均在个性化的信噪比（SNR）下进行，该信噪比约为同步AV句子正确语音识别率的70%。老年听者（无论是否有听力损失）的AV同时性窗口通常比年轻听者更宽，这可能反映了在听觉领先条件下的听觉时间处理变慢和在听觉滞后条件下的对异步性的敏感性降低。然而，老年和年轻听者在语音识别任务中受到听觉和视觉信号起始时间错位的影响相似。这表明，即使老年听者没有意识到刺激是异步的，他们也会因时间错位而对语音识别产生负面影响。总体而言，研究结果表明，当听者对同时性AV语音信号的表现相当时，年龄效应在检测判断中是显而易见的，但在异步语音的识别中并不明显。