Detection and Recognition of Asynchronous Auditory/Visual Speech: Effects of Age, Hearing Loss, and Talker Accent

This investigation examined age-related differences in auditory-visual (AV) integration as reflected on perceptual judgments of temporally misaligned AV English sentences spoken by native English and native Spanish talkers. In the detection task, it was expected that slowed auditory temporal processing of older participants, relative to younger participants, would be manifest as a shift in the range over which participants would judge asynchronous stimuli as synchronous (referred to as the “AV simultaneity window”). The older participants were also expected to exhibit greater declines in speech recognition for asynchronous AV stimuli than younger participants. Talker accent was hypothesized to influence listener performance, with older listeners exhibiting a greater narrowing of the AV simultaneity window and much poorer recognition of asynchronous AV foreign-accented speech compared to younger listeners. Participant groups included younger and older participants with normal hearing and older participants with hearing loss. Stimuli were video recordings of sentences produced by native English and native Spanish talkers. The video recordings were altered in 50 ms steps by delaying either the audio or video onset. Participants performed a detection task in which the judged whether the sentences were synchronous or asynchronous, and performed a recognition task for multiple synchronous and asynchronous conditions. Both the detection and recognition tasks were conducted at the individualized signal-to-noise ratio (SNR) corresponding to approximately 70% correct speech recognition performance for synchronous AV sentences. Older listeners with and without hearing loss generally showed wider AV simultaneity windows than younger listeners, possibly reflecting slowed auditory temporal processing in auditory lead conditions and reduced sensitivity to asynchrony in auditory lag conditions. However, older and younger listeners were affected similarly by misalignment of auditory and visual signal onsets on the speech recognition task. This suggests that older listeners are negatively impacted by temporal misalignments for speech recognition, even when they do not notice that the stimuli are asynchronous. Overall, the findings show that when listener performance is equated for simultaneous AV speech signals, age effects are apparent in detection judgments but not in recognition of asynchronous speech.

本研究考察了年龄相关的视听（auditory-visual, AV）整合差异，该差异体现在对母语英语者与母语西班牙语者录制的时间错位视听英语语句的知觉判断中。在检测任务中，相较于年轻被试，老年被试较慢的听觉时间加工（auditory temporal processing）过程，预计会表现为被试将异步刺激判断为同步刺激的范围发生偏移，该范围被称为“视听同时性窗口（AV simultaneity window）”。同时预计，相较于年轻被试，老年被试在异步视听刺激下的语音识别表现会出现更显著的下降。研究还假设，说话者口音会影响听者表现，且老年听者相比年轻听者，其视听同时性窗口会出现更明显的收窄，同时对带有外国口音的异步视听语音的识别表现会更差。被试组包括听力正常的年轻与老年被试，以及存在听力损失（hearing loss）的老年被试。刺激材料为母语英语者与母语西班牙语者录制的语句视频，该视频通过延迟音频或视频起始时间，以50毫秒为间隔进行调整。被试完成两项任务：其一为检测任务，需判断语句是否同步；其二为识别任务，针对多种同步与异步条件完成识别测试。两项任务均在个体化信噪比（signal-to-noise ratio, SNR）下进行，该信噪比对应同步视听语句约70%正确率的语音识别表现。无论是否存在听力损失，老年听者的视听同时性窗口整体上比年轻听者更宽，这可能反映了在听觉领先条件下的听觉时间加工减慢，以及在听觉滞后条件下对时间错位的敏感性降低。然而，在语音识别任务中，视听信号起始错位对老年与年轻听者的影响程度相似。这表明，即便未察觉到刺激存在时间错位，老年听者仍会因时间错位而在语音识别中受到负面影响。整体而言，研究结果显示，当听者在同步视听语音信号下的表现被标准化后，年龄效应仅在检测判断中显现，而在异步语音的识别任务中未表现出显著差异。