Artificial Intelligence-Generated Draft Replies to Patient Messages in Pediatrics

Objective: This study describes the utilization of and experiences with artificial intelligence (AI)-generated draft responses to patient messages in pediatric ambulatory clinicians and contextualizes their experiences in relation to those of adult specialty clinicians.
Materials and Methods: A prospective pilot was conducted from September 2023 to August 2024 in two pediatric clinics (General Pediatric and Adolescent Medicine) and two obstetric clinics (Reproductive Endocrinology and Infertility and General Obstetrics) within an academic health system in Northern California. Participants included physician, nurse, and medical assistant volunteers. The intervention involved a feature utilizing large language models embedded in the electronic health record to generate draft responses. Proportion of AI-generated draft used was collected, as were pre-pilot and follow-up surveys.
Results: A total of 61 clinicians (26 pediatric, 35 obstetric) enrolled, with 46 (75%) completing both surveys. Pediatric clinicians utilized 13.3% (95% CI 12.3-14.4%) of AI-generated drafts, and usage rates when responding to patients vs their proxies was similar (15% vs 12.9%, p = 0.24). Despite using AI-generated drafts significantly less than obstetric clinicians (18.3% [17.2-19.5%], p < 0.0001), pediatric clinicians reported a significant reduction in perceived task load (NASA TLX: 59.9 to 50.9, p = 0.04) and were more likely to recommend the tool (LTR: 7.0 vs. 5.2, p = 0.04).
Discussion and Conclusion: Pediatric clinicians used AI-generated drafts at a rate within previously reported ranges in adult specialties and experienced utility. These findings suggest this tool has potential for enhancing efficiency and reducing task load in pediatric care.