Preparing educational presentation about therapeutic exercises to resident physiatrists

This study evaluated the effectiveness of ChatGPT-4o, a generative artificial intelligence (AI) platform, in preparing educational presentation on therapeutic exercises specifically designed for physiatry residents. Both a physiatry expert and ChatGPT-4 created PowerPoint slides for a presentation on therapeutic exercises using the same reputable sources. Two other physiatry experts, blinded to the origin of the presentation and each other's scores, independently assessed both presentations using four of the CLEAR criteria (completeness, lack of false information, appropriateness, and relevance).

Statistical analyses confirmed the interrater reliability. The average scores of the expert-prepared slides were significantly higher than those of the AI-prepared slides. However, when assessing the presentations as a whole, no statistically significant difference emerged between the average AI and expert scores. The overall scores of the AI-prepared slides were rated between good and excellent, while the expert-prepared slides were rated as excellent. Similarly, when assessing the presentations as a whole, the AI performed good whereas expert performed excellent. Upon evaluating the variability within the model for the assessed criteria, it was found that the AI ranked highest in relevance, whereas the expert ranked highest in terms of lack of false information.

These findings indicated that although ChatGPT-4o can produce effective educational content, the expert still outperformed AI. This underscores the importance of professional oversight in maintaining the educational quality of resident physician training. Furthermore, fostering collaboration between humans and AI can lead to enhanced educational outcomes in the field.