Look at Me When I Talk to You

本研究介绍了一个名为“Look at Me When I Talk to You”的开放源视频数据集，旨在通过分析用户面部表情来识别语音助手的错误。该数据集由康奈尔科技创建，包含21名参与者与语音助手互动的视频片段，总计1238个片段。数据集中的视频从智能扬声器的视角捕捉，记录了参与者在自由形式互动中的面部反应。此数据集的应用领域广泛，主要用于训练机器学习模型，以提高语音助手在识别和自我修复错误方面的能力，从而改善人机交互体验。

This study introduces an open-source video dataset titled "Look at Me When I Talk to You", which is designed to identify errors made by voice assistants by analyzing users' facial expressions. Developed by Cornell Tech, the dataset contains video clips of 21 participants interacting with voice assistants, totaling 1238 clips. All videos are captured from the perspective of smart speakers, recording the facial reactions of participants during free-form interactions. This dataset has broad application scenarios, and is mainly used to train machine learning models to enhance the ability of voice assistants to recognize and self-repair errors, thereby improving human-computer interaction experiences.