Role of artificial intelligence in primary prevention of common musculoskeletal diseases.

We designed a mixed-methods study to evaluate the effectiveness of ChatGPT versions 3.5 and 4 in delivering primary prevention information for common chronic musculoskeletal disorders (MSDs), acknowledging the increasing global burden of these conditions.

The evaluation encompassed 15 queries targeting the most prevalent MSDs. Each query was crafted to solicit preventive measures and lifestyle changes that could mitigate the risk of developing these disorders. Designed to mirror real-life inquiries, the queries were tested on both ChatGPT versions under standard default settings to ensure reproducibility. Since both versions do not retain information from previous interactions, each session was initiated anew to avoid potential learning or feedback loops.

Responses were independently assessed by two expert raters—specialist physiatrists—who evaluated the AI models for completeness, lack of false information, evidence support, appropriateness, and relevance of the information provided. Inter-rater reliability was quantified using Cohen’s kappa statistic to ensure objective evaluation of the AI-generated content.

Statistical analyses, including descriptive statistics, Wilcoxon signed-rank tests, and paired samples t-tests, were conducted to identify and compare the performance of the two AI versions. This structured approach provided a robust framework for analyzing the capabilities of conversational AI in supporting primary prevention strategies in public health.