Supplementary Material for: Utilization of Machine Learning-Based Computer Vision and Voice Analysis to Derive Digital Biomarkers of Cognitive Functioning in Trauma Survivors

Background: Alterations in multiple domains of cognition have been observed in individuals who have experienced a traumatic stressor. These domains may provide important insights in identifying underlying neurobiological dysfunction driving an individual’s clinical response to trauma. However, such assessments are burdensome, costly, and time-consuming. To overcome barriers, efforts have emerged to measure multiple domains of cognitive functioning through the application of machine learning (ML) models to passive data sources. Methods: We utilized automated computer vision and voice analysis methods to extract facial, movement, and speech characteristics from semi-structured clinical interviews in 81 trauma survivors who additionally completed a cognitive assessment battery. A ML-based regression framework was used to identify variance in visual and auditory measures that relate to multiple cognitive domains. Results: Models derived from visual and auditory measures collectively accounted for a large variance in multiple domains of cognitive functioning, including motor coordination (R2 = 0.52), processing speed (R2 = 0.42), emotional bias (R2 = 0.52), sustained attention (R2 = 0.51), controlled attention (R2 = 0.44), cognitive flexibility (R2 = 0.43), cognitive inhibition (R2 = 0.64), and executive functioning (R2 = 0.63), consistent with the high test-retest reliability of traditional cognitive assessments. Face, voice, speech content, and movement have all significantly contributed to explaining the variance in predicting functioning in all cognitive domains. Conclusions: The results demonstrate the feasibility of automated measurement of reliable proxies of cognitive functioning through low-burden passive patient evaluations. This makes it easier to monitor cognitive functions and to intervene earlier and at a lower threshold without requiring a time-consuming neurocognitive assessment by, for instance, a licensed psychologist with specialized training in neuropsychology.

研究背景：经历创伤性应激事件的个体常出现多维度认知功能改变。此类认知维度或可为揭示驱动个体创伤临床应答的潜在神经生物学功能异常提供重要线索。然而，传统认知评估往往存在负担重、成本高且耗时冗长的局限。为破解上述瓶颈，学界已涌现出通过将机器学习（Machine Learning, ML）模型应用于被动数据源，以实现多维度认知功能测评的探索方向。

研究方法：本研究纳入81名创伤幸存者，所有受试者均完成半结构化临床访谈与全套认知评估。研究采用自动化计算机视觉与语音分析技术，从半结构化临床访谈素材中提取面部、肢体动作与言语特征；随后构建基于机器学习的回归框架，以识别与多维度认知功能相关的视觉与听觉测量指标变异量。

研究结果：基于视觉与听觉指标构建的模型，可解释多维度认知功能的大量变异量，涵盖运动协调（R²=0.52）、加工速度（R²=0.42）、情绪偏向（R²=0.52）、持续性注意（R²=0.51）、控制性注意（R²=0.44）、认知灵活性（R²=0.43）、认知抑制（R²=0.64）与执行功能（R²=0.63），该结果与传统认知评估的高重测信度相符。面部、语音、言语内容与肢体动作均对所有认知维度的功能预测变异量解释具有显著贡献。

研究结论：本研究结果证实，通过低负担的被动患者评估，可实现认知功能可靠替代指标的自动化测评。该方法无需耗时的神经认知评估——例如由接受过神经心理学专项培训的执业心理学家完成的评估——即可更便捷地实现认知功能监测，并以更低阈值开展早期干预。