Performance of large language models on Thailand’s national medical licensing examination: a cross-sectional study

This study aimed to evaluate the feasibility of general-purpose large language models (LLMs) in addressing inequities in medical licensure exam preparation for Thailand’s National Medical Licensing Examination (ThaiNLE), which currently lacks standardized public study materials. We assessed four multi-modal LLMs (GPT-4, Claude 3 Opus, Gemini 1.0/1.5 Pro) using a 304-question ThaiNLE Step 1 mock examination (10.2% image-based), applying deterministic API configurations and five inference repetitions per model. Performance was measured via micro- and macro-accuracy metrics compared against historical passing thresholds.