Conveying facial expressions to blind and visually impaired persons through a wearable vibrotactile device

In face-to-face social interactions, blind and visually impaired persons (VIPs) lack access to nonverbal cues like facial expressions, body posture, and gestures, which may lead to impaired interpersonal communication. In this study, a wearable sensory substitution device (SSD) consisting of a head mounted camera and a haptic belt was evaluated to determine whether vibrotactile cues around the waist could be used to convey facial expressions to users and whether such a device is desired by VIPs for use in daily living situations. Ten VIPs (mean age: 38.8, SD: 14.4) and 10 sighted persons (SPs) (mean age: 44.5, SD: 19.6) participated in the study, in which validated sets of pictures, silent videos, and videos with audio of facial expressions were presented to the participant. A control measurement was first performed to determine how accurately participants could identify facial expressions while relying on their functional senses. After a short training, participants were asked to determine facial expressions while wearing the emotion feedback system. VIPs using the device showed significant improvements in their ability to determine which facial expressions were shown. A significant increase in accuracy of 44.4% was found across all types of stimuli when comparing the scores of the control (mean±SEM: 35.0±2.5%) and supported (mean±SEM: 79.4±2.1%) phases. The greatest improvements achieved with the support of the SSD were found for silent stimuli (68.3% for pictures and 50.8% for silent videos). SPs also showed consistent, though not statistically significant, improvements while supported. Overall, our study shows that vibrotactile cues are well suited to convey facial expressions to VIPs in real-time. Participants became skilled with the device after a short training session. Further testing and development of the SSD is required to improve its accuracy and aesthetics for potential daily use.

在面对面社交互动中，盲人与视觉障碍人士（Visually Impaired Persons, VIPs）无法获取面部表情、身体姿态与手势等非语言线索，这可能会损害其人际沟通能力。本研究对一款由头戴式摄像头与触觉腰带组成的可穿戴感官替代设备（Sensory Substitution Device, SSD）展开评估，旨在探究腰部周围的振动触觉线索能否用于向使用者传递面部表情，以及该设备是否符合VIPs日常使用的需求。本研究共有10名VIPs（平均年龄38.8岁，标准差SD=14.4）与10名视力正常者（Sighted Persons, SPs，平均年龄44.5岁，标准差SD=19.6）参与。实验中向参与者展示了经过验证的面部表情图片组、无声视频组以及带音频的面部表情视频组。实验首先开展对照测量，以确定参与者在仅依靠自身感官的情况下，识别面部表情的准确率。经过简短培训后，要求参与者佩戴该设备，完成面部表情识别任务。使用该设备的VIPs在识别面部表情的能力上表现出显著提升。对比对照阶段（平均值±标准误SEM：35.0±2.5%）与设备辅助阶段（平均值±标准误SEM：79.4±2.1%）的得分，所有类型刺激的识别准确率均提升了44.4%，提升效果具有统计学显著性。在SSD辅助下，识别准确率提升最为显著的为无声刺激：图片类提升68.3%，无声视频类提升50.8%。视力正常者在设备辅助下也展现出一致的提升效果，但该提升未达到统计学显著性水平。总体而言，本研究证实振动触觉线索能够很好地向VIPs实时传递面部表情信息。参与者仅需经过简短培训即可熟练使用该设备。未来仍需对SSD开展进一步测试与优化，以提升其识别准确率与外观设计，使其更适配日常使用场景。