Data_Sheet_2_“Alexa, You're Really Stupid”: A Longitudinal Field Study on Communication Breakdowns Between Family Members and a Voice Assistant.PDF

In this paper, we present the results of our long-term study on use of a voice assistant (Amazon Alexa via Amazon Echo Dot) in nine families with children and no previous experience with this technology. The study was conducted over the course of 5 weeks during which the families could interact with the device freely. Three house visits were made to collect empirical data from the adult participants in form of questionnaires. Additionally, conversational data from log files of the voice assistant were obtained. These data were annotated and analyzed with a focus on communication breakdowns during human-assistant interaction. We investigate user behavior for both adults and children in such situations, its reasons and consequences for user satisfaction. This article provides qualitative analysis of three particularly interesting breakdown cases, as well as statistical analysis along several hypotheses and research questions combining empirical and conversational data. Described cases of communication breakdown illustrate findings from existing literature on the topic. The statistical analysis paints a mixed picture, however, it helped us identify further avenues for research, some of which can be explored with our data set in the future. We found a significant negative effect of the number of abandoned failed requests on user satisfaction, contrary to the number of successfully repaired requests that had no influence on user satisfaction. We discovered that users are more inclined to use reformulation as repair strategy when they do not perceive the emergence of miscommunication as their fault. We could not identify a significant effect of internal reasons for the choice of other strategies, so we suggest that situational clues such as the immediate response of the voice assistant are more important for the choice of repair strategy. Our results also hint that users distinguish between repair strategies differently, as the self-perceived frequency of repetitions and abortions of requests were found to be positive predictors for the use of reformulation-based strategies. With regards to the long-term aspect of the study, use of repetition as a repair strategy by both children and adults significantly decreased with time, no other changes were found for other strategies. Additionally, no significant impact of age on the choice of repair strategy was found, as well as no interaction effect between age and time.

本文汇报了一项长期研究的成果：针对9个有儿童且此前未接触过此类语音助手（voice assistant）技术的家庭，调研其使用搭载亚马逊回声点（Amazon Echo Dot）的亚马逊艾拉（Amazon Alexa）语音助手的情况。本次调研周期为5周，期间受试家庭可自由与该设备交互。研究人员共开展3次家访，以问卷形式从成年参与者处收集实证数据；此外还获取了该语音助手日志文件中的对话数据。上述数据均经过标注与分析，重点关注人机助手交互过程中的沟通中断（communication breakdowns）问题。本研究探究了此类场景下成人与儿童的用户行为、行为成因及其对用户满意度的影响。本文针对3个典型沟通中断案例展开质性分析，并结合实证数据与对话数据，围绕多项假说与研究问题开展统计分析。文中提及的沟通中断案例，可佐证该领域现有文献的相关研究结论。统计分析结果呈现出复杂态势，但本研究借此明确了后续研究方向，其中部分方向可依托本次数据集在未来展开探索。研究发现，未完成的失败请求数量对用户满意度存在显著负向影响；而成功修复的请求数量则对用户满意度无显著影响，二者形成鲜明对照。研究发现，当用户未将沟通误解的成因归咎于自身时，更倾向于采用重构（reformulation）作为修复策略。本研究未发现用户选择其他修复策略的内在动因存在显著影响，因此我们认为，语音助手的即时响应等情境线索，对修复策略的选择更为关键。研究结果还表明，用户对不同修复策略的区分方式存在差异：用户自我报告的请求重复与中止频率，可作为基于重构的修复策略使用情况的正向预测因子。就调研的长期维度而言，成人与儿童均随时间推移显著减少了将重复作为修复策略的使用频次，其余修复策略则未出现显著变化。此外，本研究未发现年龄对修复策略的选择存在显著影响，也未发现年龄与时间存在交互效应。