Data underlying the publication: Timing, Indicators, and Approaches to Digital Patient Experience Evaluation: Umbrella Systematic Review

<strong>Background:</strong>The increasing prevalence of DH applications has outpaced research and practice in digital health (DH) evaluations. Patient experience (PEx) was reported as one of the challenges facing the health system by the World Health Organization. To generate evidence on DH and promote the appropriate integration and use of technologies, a standard evaluation of PEx in DH is required.<strong>Objective:</strong>This study aims to systematically identify evaluation timing considerations (ie, <em>when to measure</em>), evaluation indicators (ie, <em>what to measure</em>), and evaluation approaches (ie, <em>how to measure</em>) with regard to digital PEx. The overall aim of this study is to generate an evaluation guide for further improving digital PEx evaluation.<strong>Methods:</strong>This is a 2-phase study parallel to our previous study. In phase 1, literature reviews related to PEx in DH were systematically searched from Scopus, PubMed, and Web of Science databases. Two independent raters conducted 2 rounds of paper screening, including title and abstract screening and full-text screening, and assessed the interrater reliability for 20% (round 1: 23/115 and round 2: 12/58) random samples using the Fleiss-Cohen coefficient (round 1: <em>k</em>1=0.88 and round 2: <em>k</em>2=0.80). When reaching interrater reliability (<em>k</em>&gt;0.60), TW conducted the rest of the screening process, leaving any uncertainties for group discussions. Overall, 38% (45/119) of the articles were considered eligible for further thematic analysis. In phase 2, to check if there were any meaningful novel insights that would change our conclusions, we performed an updated literature search in which we collected 294 newly published reviews, of which 102 (34.7%) were identified as eligible articles. We considered them to have no important changes to our original results on the research objectives. Therefore, they were not integrated into the synthesis of this review and were used as supplementary materials.<strong>Results:</strong>Our review highlights 5 typical evaluation objectives that serve 5 stakeholder groups separately. We identified a set of key evaluation timing considerations and classified them into 3 categories: intervention maturity stages, timing of the evaluation, and timing of data collection. Information on evaluation indicators of digital PEx was identified and summarized into 3 categories (intervention outputs, patient outcomes, and health care system impact), 9 themes, and 22 subthemes. A set of evaluation theories, common study designs, data collection methods and instruments, and data analysis approaches was captured, which can be used or adapted to evaluate digital PEx.<strong>Conclusions:</strong>Our findings enabled us to generate an evaluation guide to help DH intervention researchers, designers, developers, and program evaluators evaluate digital PEx. Finally, we propose 6 directions for encouraging further digital PEx evaluation research and practice to address the challenge of poor PEx.

背景：数字健康（Digital Health, DH）应用的普及率不断提升，但其相关评估的研究与实践却相对滞后。世界卫生组织指出，患者体验（Patient Experience, PEx）是当前卫生系统面临的挑战之一。为了获取数字健康领域的相关研究证据，并推动技术的合理整合与应用，亟需建立针对数字健康场景下患者体验的标准化评估体系。
研究目标：本研究旨在系统梳理数字健康患者体验（Digital PEx）评估的核心考量维度，即**何时开展评估**、**评估指标为何**以及**如何实施评估**。本研究的最终目标是形成一套标准化评估指南，以助力完善数字健康患者体验的评估工作。
研究方法：本研究为两阶段设计，与本团队此前的研究并行开展。第一阶段：系统检索Scopus、PubMed及Web of Science三大数据库中与数字健康患者体验相关的文献综述。由两名独立评分员开展两轮文献筛选：首轮为标题与摘要筛选，次轮为全文筛选；同时采用Fleiss-Cohen系数，分别对第一轮（23/115，占比20%）、第二轮（12/58，占比20%）的随机抽样样本进行组间信度检验，两轮的kappa值分别为*k*₁=0.88与*k*₂=0.80。当组间信度达到*k*>0.60的标准后，由TW完成剩余的文献筛选工作，若存在争议则通过小组讨论解决。最终共有38%（45/119）的文献符合纳入标准，进入后续的主题分析环节。第二阶段：为验证是否存在能够改变本研究结论的重要新发现，团队开展了更新版文献检索，共获取294篇新发表的综述文献，其中102篇（占比34.7%）符合纳入标准。经评估，该部分文献未对本研究既定目标下的原始结论产生实质性影响，因此未纳入本次综述的综合分析环节，仅作为补充材料留存。
研究结果：本综述明确了五类分别面向不同利益相关群体的典型评估目标；梳理得到一系列核心评估时机考量因素，并将其划分为三大类别：干预成熟度阶段、评估实施时机与数据收集时机。同时提取得到数字健康患者体验的评估指标体系，并将其归纳为3大类（干预产出、患者结局与卫生系统影响）、9个主题以及22个子主题。此外，本研究还整理得到一套可直接使用或经调整后用于数字健康患者体验评估的理论框架、常用研究设计方案、数据收集方法与工具，以及数据分析策略。
研究结论：基于上述研究结果，本团队最终形成一套数字健康患者体验评估指南，可助力数字健康干预的研究者、设计者、开发者以及项目评估人员开展相关评估工作。最后，本研究针对当前患者体验不佳的行业痛点，提出了六大未来研究方向，以推动数字健康患者体验评估领域的后续研究与实践发展。