A Realistic Rwandan Communty Health Worker Generated Vingette-based (I.e., Open-ended Questions) Benchmarking Dataset (with Associated Clinician and LLM Responses)

The full dataset of 5,609 clinical vignettes contributed by 101 community health workers (CHWs) across four Rwandan districts is available upon request from The Centre for the Fourth Industrial Revolution, the innovation lab for the Rwandan Government. Prospective users should contact ‘[e-mail]’ to request access. This subset of 524 questions, answers, and individual evaluation results comprises the data used in a benchmarking study to compare responses generated by five large language models (LLMs) (Gemini-2, GPT-4o, o3 mini, Deepseek R1, and Meditron-70B) with those from local clinicians. The results were generated by applying a rubric of 11 expert-rated metrics, each scored on a five-point Likert scale. The associated benchmarking study, titled 'How Good Are Large Language Models at Supporting Frontline Healthcare Workers in Low-Resource Settings – A Benchmarking Study & Dataset,' contains detailed descriptions of how this data was generated and the evaluations conducted.