Data Sheet 1_Evaluating multimodal ChatGPT for emergency decision-making of ocular trauma cases.docx

PurposeThis study aimed to evaluate the potential of ChatGPT in diagnosing ocular trauma cases in emergency settings and determining the necessity for surgical intervention.
MethodsThis retrospective observational study analyzed 52 ocular trauma cases from Ningbo Eye Hospital. Each case was input into GPT-3.5 turbo and GPT-4.0 turbo in Chinese and English. Ocular surface photographs were independently incorporated into the input to assess ChatGPT’s multimodal performance. Six senior ophthalmologists evaluated the image descriptions generated by GPT-4.0 turbo.
ResultsWith text-only input, the diagnostic accuracy rate was 80.77%–88.46% with GPT-3.5 turbo and 94.23%–98.08% with GPT-4.0 turbo. After replacing examination information with photography, GPT-4.0 turbo’s diagnostic accuracy rate decreased to 63.46%. In the image understanding evaluation, the mean completeness scores attained 3.59 ± 0.94 to 3.69 ± 0.90. The mean correctness scores attained 3.21 ± 1.04 to 3.38 ± 1.00.
ConclusionThis study demonstrates ChatGPT has the potential to help emergency physicians assess and triage ocular trauma patients properly and timely. However, its ability in clinical image understanding needs to be further improved.