Data underlying the publication: Timing, Indicators, and Approaches to Digital Patient Experience Evaluation: Umbrella Systematic Review

<strong>Background:</strong>The increasing prevalence of DH applications has outpaced research and practice in digital health (DH) evaluations. Patient experience (PEx) was reported as one of the challenges facing the health system by the World Health Organization. To generate evidence on DH and promote the appropriate integration and use of technologies, a standard evaluation of PEx in DH is required.<strong>Objective:</strong>This study aims to systematically identify evaluation timing considerations (ie, <em>when to measure</em>), evaluation indicators (ie, <em>what to measure</em>), and evaluation approaches (ie, <em>how to measure</em>) with regard to digital PEx. The overall aim of this study is to generate an evaluation guide for further improving digital PEx evaluation.<strong>Methods:</strong>This is a 2-phase study parallel to our previous study. In phase 1, literature reviews related to PEx in DH were systematically searched from Scopus, PubMed, and Web of Science databases. Two independent raters conducted 2 rounds of paper screening, including title and abstract screening and full-text screening, and assessed the interrater reliability for 20% (round 1: 23/115 and round 2: 12/58) random samples using the Fleiss-Cohen coefficient (round 1: <em>k</em>1=0.88 and round 2: <em>k</em>2=0.80). When reaching interrater reliability (<em>k</em>&gt;0.60), TW conducted the rest of the screening process, leaving any uncertainties for group discussions. Overall, 38% (45/119) of the articles were considered eligible for further thematic analysis. In phase 2, to check if there were any meaningful novel insights that would change our conclusions, we performed an updated literature search in which we collected 294 newly published reviews, of which 102 (34.7%) were identified as eligible articles. We considered them to have no important changes to our original results on the research objectives. Therefore, they were not integrated into the synthesis of this review and were used as supplementary materials.<strong>Results:</strong>Our review highlights 5 typical evaluation objectives that serve 5 stakeholder groups separately. We identified a set of key evaluation timing considerations and classified them into 3 categories: intervention maturity stages, timing of the evaluation, and timing of data collection. Information on evaluation indicators of digital PEx was identified and summarized into 3 categories (intervention outputs, patient outcomes, and health care system impact), 9 themes, and 22 subthemes. A set of evaluation theories, common study designs, data collection methods and instruments, and data analysis approaches was captured, which can be used or adapted to evaluate digital PEx.<strong>Conclusions:</strong>Our findings enabled us to generate an evaluation guide to help DH intervention researchers, designers, developers, and program evaluators evaluate digital PEx. Finally, we propose 6 directions for encouraging further digital PEx evaluation research and practice to address the challenge of poor PEx.

背景：数字健康（Digital Health，DH）应用的日益普及，其发展速度已超过数字健康评估领域的研究与实践。患者体验（Patient Experience，PEx）被世界卫生组织（World Health Organization）列为卫生系统面临的挑战之一。为了生成数字健康相关证据并推动技术的合理整合与应用，亟需建立针对数字健康场景下患者体验的标准化评估体系。
目的：本研究旨在系统梳理数字健康场景下患者体验的评估时机考量（即「何时实施评估」）、评估指标（即「评估哪些内容」）以及评估方法（即「如何开展评估」）。本研究的总体目标是形成一套评估指南，以进一步完善数字健康患者体验评估工作。
方法：本研究为与前期研究并行开展的两阶段研究。第一阶段：从Scopus、PubMed、Web of Science数据库中系统检索与数字健康患者体验相关的文献。由2名独立评阅人开展2轮文献筛选：首先进行标题与摘要筛选，随后进行全文筛选；并针对20%的随机样本（第一轮：115篇中的23篇，第二轮：58篇中的12篇）采用Fleiss-Cohen系数进行评阅者间信度检验（第一轮κ₁=0.88，第二轮κ₂=0.80）。当评阅者间信度达标（κ>0.60）后，由TW完成剩余筛选流程，对存在疑问的内容组织小组讨论。最终共有38%（119篇中的45篇）的文献符合主题分析的纳入标准。第二阶段：为验证是否存在可改变研究结论的有价值新发现，我们开展了更新版文献检索，共收集到294篇新发表的综述类文献，其中102篇（34.7%）被判定为符合纳入标准。经评估，该部分文献未对本研究的核心研究结论产生重要影响，因此未纳入本次综述的综合分析环节，仅作为补充材料留存。
结果：本综述明确了5类分别面向不同利益相关群体的典型评估目标。我们梳理出一系列关键的评估时机考量因素，并将其划分为3大类：干预成熟度阶段、评估实施时机以及数据采集时机。针对数字健康患者体验的评估指标信息，我们经归纳总结后分为3大类（干预产出、患者结局以及卫生保健系统影响）、9个主题以及22个子主题。此外，本研究还梳理了一系列可用于或适配于数字健康患者体验评估的评估理论、常用研究设计、数据采集方法与工具以及数据分析方法。
结论：本研究的研究成果可用于生成一套评估指南，以帮助数字健康干预的研究人员、设计师、开发者以及项目评估人员开展数字健康患者体验评估工作。最后，我们提出了6个未来研究方向，旨在推动进一步的数字健康患者体验评估研究与实践，以应对患者体验不佳的挑战。