DATASET FOR A Framework for Evaluating Dashboards in Healthcare

This dataset is the supporting material for the manuscript titled A Framework for Evaluating Dashboards in Healthcare (undr-review of IEEE Transactions on Visualization and Computer Graphics). Abstract: In the era of ‘information overload’, effective information provision is essential for enabling rapid response and critical decision making. In making sense of diverse information sources, data dashboards have become an indispensable tool, providing fast, effective, adaptable, and personalized access to information for professionals and the general public alike. However, these objectives place a heavy requirement on dashboards as information systems, in usability and ineffective design. Understanding these shortfalls is a challenge given the absence of a consistent and comprehensive approach to dashboard evaluation. In this paper we systematically review literature on dashboard implementation in the healthcare domain, a field where dashboards have been employed widely, and in which there is widespread interest for improving the current state of the art, and subsequently analyse approaches taken towards evaluation. We draw upon consolidated dashboard literature and our own observations to introduce a general definition of dashboards which is more relevant to current trends, together with seven evaluation scenarios - task performance, behaviour change, interaction workflow, perceived engagement, potential utility, algorithm performance and system implementation. These scenarios distinguish different evaluation purposes which we illustrate through measurements, example studies, and common challenges in evaluation study design. We provide a breakdown of each evaluation scenario, and highlight some of the subtle and less well posed questions. We demonstrate the use of proposed framework by a design study guided by this framework. We conclude by comparing this framework with existing literature, outlining a number of active discussion points and a set of dashboard evaluation best practices for the academic, clinical and software development communities alike.