Adopting Maximum Pupil Diameter to Detect Subtle Usability Issues of a Smartphone Application, Conflict Solver

With the increasing popularity and unparalleled ubiquity of smartphones, researchers in various fields are designing and developing novel applications for this platform as part of the solution to the challenging problems they face. Usability lies at the core of the user experience of such applications. However, the assessments used are often limited to summative and post-event methods, which can overlook subtle yet impactful issues. Objective and instantaneous measures of cognitive workloads provide a solution to this shortcoming. Our previous research has established the reliability of maximum pupil dilation, measured with Tobii Pro Nano, as a preeminent indicator of cognitive workload surges in mobile application users. In this study, we used this measure to locate user cognitive workload peaks while using Conflict Solver and discovered subtle user interface issues that were not reported in the post-usability interview. A total of 30 participants completed a Conflict Solver usability experiment with two phases. In phase 1, the participants performed two “Add a Term” tasks on the original Conflict Solver, followed by a semi-structured interview about their experience with the application. A few subtle usability issues with a drop-down menu were detected through identifying user cognitive workload peaks. In phase 2, the same participants completed the same tasks on Conflict Solver with a redesigned and extended drop-down menu. The results showed that the new design solved the usability issues, and the participants became more favor the drop-down menu over the input box. In conclusion, including maximum pupil dilation into the usability assessment toolkit would provide a more objective and comprehensive usability assessment of a smartphone application. It can also be used to verify the successfulness of a user interface design solution.随着智能手机日益普及且覆盖范围空前广泛，各领域研究者正针对该平台开发新型应用，以此作为解决其所面临棘手问题的方案之一。可用性（usability）是此类应用用户体验的核心所在，但当前所采用的评估手段往往局限于总结式与事后评估方法（summative and post-event methods），易忽略那些虽细微却影响显著的问题。客观且即时的认知负荷（cognitive workload）测量方法，正是弥补这一缺陷的可行方案。我们此前的研究已证实，通过Tobii Pro Nano眼动仪测得的最大瞳孔扩张幅度（maximum pupil dilation），可作为移动应用用户认知负荷激增的卓越指标。本研究中，我们借助该测量手段定位了用户使用Conflict Solver时的认知负荷峰值，并发现了事后可用性访谈中未被提及的细微用户界面（user interface）问题。共有30名受试者参与了分两阶段开展的Conflict Solver可用性实验。第一阶段中，受试者先在原版Conflict Solver上完成两项"Add a Term"任务，随后接受关于该应用使用体验的半结构化访谈（semi-structured interview）。通过识别用户的认知负荷峰值，我们发现了该应用下拉菜单（drop-down menu）存在的几处细微可用性问题。第二阶段中，同一批受试者在经过重新设计与扩展的下拉菜单版Conflict Solver上完成了相同任务。结果显示，新设计成功解决了此前的可用性问题，且受试者相较于输入框，更青睐下拉菜单。综上，将最大瞳孔扩张幅度纳入可用性评估工具集，可为智能手机应用提供更为客观且全面的可用性评估，同时也可用于验证用户界面设计方案的有效性。