Comparative Analysis of Artificial Intelligence Platforms: GPT-4 and Google Gemini in Answering Questions about Birth Control Methods

AbstractBackground: Birth control methods (BCMs) are often underutilized or misunderstood, especially among young individuals entering their reproductive years. With the growing reliance on artificial intelligence (AI) platforms for health-related information, this study evaluates the performance of GPT-4 (OpenAI, San Francisco, CA, USA) and Google Gemini (Google, Mountain View, CA, USA) in addressing commonly asked questions about BCMs.Methods: Thirty questions, derived from the American College of Obstetrics and Gynecologists website, were posed to both AI platforms. Questions spanned four categories: general contraception, specific contraceptive types, emergency contraception, and other topics. Responses were evaluated using a 5-point rubric assessing accuracy, completeness, and lack of false information. Overall scores were calculated by averaging the rubric scores. Statistical analysis, including the Wilcoxon signed-rank and Kruskal-Wallis tests, was performed to compare performance metrics.Results: ChatGPT and Google Gemini both provided high-quality responses, with overall scores averaging 4.38 ± 0.58 and 4.37 ± 0.52, respectively, categorized as "excellent." ChatGPT outperformed in reducing false information (4.70 ± 0.60 vs. 4.47 ± 0.73), while Google Gemini excelled in accuracy (4.53 ± 0.57 vs. 4.30 ± 0.70). Completeness scores were comparable. No significant differences were found in overall performance (p = 0.548), though Google Gemini showed a significant edge in accuracy (p = 0.035). Both platforms scored consistently across question categories, with no statistically significant differences noted.Conclusions: GPT-4 and Google Gemini provide reliable and accurate responses to BCM-related queries, with slight differences in strengths. These findings underscore the potential of AI tools in addressing public health information needs, particularly for young individuals seeking guidance on contraception. Further studies with larger datasets may elucidate nuanced differences between AI platforms.