Artificial Social Agent Questionnaire Instrument

We present the Artificial Social Agent Questionnaire (ASAQ), an instrument for evaluating human interaction with an artificial social agent (ASA), resulted from multi-year efforts involving more than 100 Intelligent Virtual Agent (IVA) researchers worldwide within the OSF work-group of Artificial Social Agent Evaluation Instrument (https://osf.io/6duf7/). It has 19 measurement constructs constituted by 90 items, which capture more than 80% of the constructs identified in empirical studies published in the IVA conference 2013-2018. In addition, a representative item of each construct or dimension was selected to create a 24-item short version of the ASA questionnaire. Whereas the long version is suitable for a comprehensive evaluation of human-ASA interaction, the short version allows quick analysis and description of the interaction with the ASA. To support reporting ASAQ results, we also put forward two ASAQ charts. The charts provides a quick overview of agent profile. The ASAQ Chart can be used for comparing the ASAQ results of up-to 4 ASAs on the original -3 to 3 scale, while the ASAQ Percentile Chart can be used for contrasting the ASAQ results with an ASAQ Representative Set. This set contains dataset of representative ASAs and their unique participants' ASAQ ratings.本研究提出人工社会智能体问卷（Artificial Social Agent Questionnaire，ASAQ），一款用于评估人类与人工社会智能体（artificial social agent，ASA）交互体验的标准化测评工具。该问卷历经多年研发，由全球超100名智能虚拟智能体（Intelligent Virtual Agent，IVA）领域研究者在人工社会智能体评估工具OSF工作组（https://osf.io/6duf7/）框架下协作完成。完整版ASAQ包含19个测量构念，共计90个测题，覆盖了2013至2018年IVA学术会议发表的实证研究中80%以上的核心测量构念。此外，研究团队从每个构念或维度中选取代表性测题，构建了包含24个测题的ASAQ精简版问卷。完整版问卷适用于对人-ASA交互进行全面评估，而精简版则可快速完成人-ASA交互的分析与描述。为便于ASAQ结果的汇报呈现，本研究同时推出两款ASAQ可视化图表，可快速直观地展示智能体的交互画像。其中，ASAQ标准图表可基于原始-3至3的计分区间，对比最多4个ASA的测评结果；而ASAQ百分位图表则可将待测ASA的测评结果与ASAQ代表性基准集进行比对。该基准集包含代表性ASA的数据集及其对应受试群体的专属ASAQ评分数据。