Morning-a-month Website Usability Testing for a Materials Science Gateway

Science gateways, offering access to scientific tools and large amounts of data in the form of web portals and other applications, are often run by the research groups who produce these tools and data. As the prevalence of these gateways increases, a key issue that arises is that of usability. The developers of gateways want users to be able to interact with the gateways easily and efficiently. However, design choices are typically made by members of the research groups involved and can be ad-hoc, biased, and generally unpredictable. To ensure users can make full use of a gateway, the need arises for a usability study.<br>The resources needed to do a large-scale usability study are quite significant. Ideal usability testing would involve comparing multiple versions of a user interface to see which version users prefer. In addition to the time and effort needed to produce these versions, many participants need to be found, scripts need to be written, and a large amount of time needs to be dedicated for the testing session itself. <br>While the benefits of large-scale usability testing are obvious, research groups often have neither the user numbers nor the rate of development that would warrant such an effort. Additionally, these groups often do not have the resources needed to fund/staff such an endeavor. <br>When starting a usability testing initiative at the Materials Project, our approach was to make usability testing as easy as possible while still obtaining valuable information. We focused on a ``one morning a month'' model and held 3 testing sessions over the course of 3 months, with 10 participants in total. Each session made use of a script to keep participant experience consistent. Scripts were slightly tweaked between sessions to obtain information on the impact of the script itself on user behavior. Each session was streamed live to observers in another room as well as recorded for later review, and observations were noted immediately after each session was over. With only 10 participants and a minimal budget, we were able to draw conclusions about web design and user behavior specific to our portal. We hope these conclusions, as well as our notes on the testing process itself, prove useful to developers of other science gateways.科学网关（Science gateways）以Web门户及其他应用程序的形式提供科学工具与海量数据访问服务，通常由开发这些工具和数据的研究团队运营。随着这类网关的普及率不断提升，可用性问题成为核心关切。网关开发者期望用户能够便捷高效地与网关交互，但设计决策往往由相关研究团队成员制定，常带有临时性、偏向性且整体难以预判。为确保用户能够充分利用网关，开展可用性研究的需求应运而生。 开展大规模可用性研究所需的资源投入相当可观。理想的可用性测试需要对比多套用户界面版本，以甄别用户偏好的方案。除了制作这些版本所需的时间与人力成本外，还需招募大量受试者、编写测试脚本，并且需要投入大量时间完成测试环节本身。 尽管大规模可用性测试的益处显而易见，但研究团队往往既缺乏足够的用户基数，也没有足够快的开发速度来支撑这类工作。此外，这类团队通常也无法为该项目提供足够的资金或人员配置。 在材料项目（Materials Project）启动可用性测试工作时，我们的思路是在尽可能获取有价值信息的前提下，将可用性测试流程简化至最便捷的程度。我们采用了“每月一个上午”的模式，在3个月内开展了3次测试，总计招募10名受试者。每次测试均使用统一脚本，以确保受试者的体验保持一致。各次测试间会对脚本进行小幅调整，以探究脚本本身对用户行为的影响。每场测试均会向另一房间的观察员进行实时直播，并录制视频供后续复盘，测试结束后会立即记录观察结果。仅用10名受试者与极低的预算，我们便得出了针对本门户的网页设计与用户行为的相关结论。我们希望这些结论，以及我们对测试流程的相关记录，能够为其他科学网关的开发者提供参考价值。