Partner-directed gaze and co-speech hand gestures: effects of age, hearing loss and noise

Data supporting the publication Partner-directed gaze and co-speech hand gestures: effects of age, hearing loss and noise.

Research on the adaptations talkers make to different communication conditions during interactive conversations has primarily focused on speech signals. We extended this type of investigation to two other important communicative signals, i.e., partner-directed gaze and iconic co-speech hand gestures with the aim of determining if the adaptations made by older adults differ from younger adults across communication conditions. We recruited 57 pairs of participants, comprising 57 primary talkers and 57 secondary ones. Primary talkers consisted of three groups: 19 older adults with mild Hearing Loss (older adult-HL); 17 older adults with Normal Hearing (older adult-NH); and 21 younger adults. The DiapixUK “spot the difference” conversation-based task was used to elicit conversions in participant pairs. One easy (No Barrier: NB) and three difficult communication conditions were tested. The three conditions consisted of two in which the primary talker could hear clearly, but the secondary talkers could not, due to multi-talker babble noise (BAB1) or a less familiar hearing loss simulation (HLS), and a condition in which both the primary and secondary talkers heard each other in babble noise (BAB2). For primary talkers, we measured mean number of partner-directed gazes; mean total gaze duration; and the mean number of co-speech hand gestures. We found a robust effects of communication condition that interacted with participant group. Effects of age were found for both gaze and gesture in BAB1, i.e., older adult-NH looked and gestured less than younger adults did when the secondary talker experienced babble noise. For hearing status, a difference in gaze between older adult-NH and older adult-HL was found for the BAB1 condition; for gesture this difference was significant in all three difficult communication conditions (older adult-HL gazed and gestured more). We propose the age effect may be due to a decline in older adult’s attention to cues signaling how well a conversation is progressing. To explain the hearing status effect, we suggest that older adult’s attentional decline is offset by hearing loss because these participants have learned to pay greater attention to visual cues for understanding speech.
Data stored on https://osf.io/4rwxq/.

本数据集为论文《以同伴为目标的注视与伴随言语的手势：年龄、听力损失与噪音的影响》提供研究数据。
此前针对交互式对话中说话者针对不同沟通情境做出的适应性调整的研究，主要聚焦于语音信号。本研究将此类调查拓展至另外两类重要的交际信号，即面向同伴的注视（partner-directed gaze）与具象伴随言语手势（iconic co-speech hand gestures），旨在探究不同沟通情境下，老年人的适应性调整是否与年轻人存在差异。
本研究招募了57组参与者，包含57名主说话者与57名副说话者。主说话者分为三组：19名轻度听力损失（hearing loss, HL）老年人、17名听力正常（normal hearing, NH）老年人，以及21名年轻人。
本研究采用DiapixUK的“找不同”对话式任务来引发参与者组内的对话行为。测试设置了1种简单情境（无屏障：No Barrier, NB）与3种复杂沟通情境。3种复杂情境中，2种情境下主说话者可清晰听见声音，但副说话者因多说话者背景噪音（multi-talker babble noise, BAB1）或不熟悉的听力损失模拟情境（hearing loss simulation, HLS）而无法清晰听见；剩余1种情境下，主、副说话者均可在混响噪音中听见彼此的声音（BAB2）。
针对主说话者，我们测量了三项指标：面向同伴的平均注视次数、平均总注视时长，以及伴随言语的手势平均次数。
研究发现，沟通情境对被试行为存在稳健效应，且该效应与参与者组别存在交互作用。在BAB1情境中，年龄对注视与手势行为均存在显著影响：当副说话者暴露于混响噪音时，听力正常老年人的注视与手势频次均低于年轻人。
针对听力状态，在BAB1情境中，听力正常老年人与轻度听力损失老年人的注视行为存在差异；而在全部3种复杂沟通情境中，二者的手势行为差异均具有统计学意义（轻度听力损失老年人的注视与手势频次更高）。
我们推测，年龄效应可能源于老年人对指示对话进展情况的线索的注意力下降。为解释听力状态效应，我们提出：老年人的注意力衰退可因听力损失得到补偿——此类参与者已学会更加关注视觉线索以辅助言语理解。
相关数据存储于https://osf.io/4rwxq/。