Performance of Large Language Models as a Tool for Primary Care Consultations

Abstract We present two datasets. On the one hand, a set of health-related queries from real users. This dataset was extracted from the 2025 Google report and validated by a panel of physicians from different specialties. It represents the primary care queries most frequently made worldwide in the English language. On the other hand, we present a set of responses generated by different Large Language Models for these queries, annotated by a panel of physicians distinct from the previous one. These responses were evaluated across several dimensions, such as medical consensus, correctness, and potential harm. The models used to generate these responses were: an open model (Llama3, version llama3:8b-instruct-q4_0), a widely used proprietary model (GPT-4, version o-mini), and an open model specialized through clinical fine-tuning (MedLlama3, version llama3-med42-8b:latest). Citation information If you use this version of the dataset, please cite as follows: Fernández-Pichel, M., Pascual Presa, N., Losada, D. E., García Orosa, B., Gude, F., Costa Lathan, C., Sueiro Justel, J., Gómez Fontenla, A., Lastra Pérez, M., & Alonso García, F. (2026). Performance of Large Language Models as a Tool for Primary Care Consultations [Data set]. Zenodo. https://doi.org/10.5281/zenodo.18311708 Acknowledgements This dataset is part of the R&D project Artificial Intelligence in Digital Media in Spain: Effects and Roles ( PID2024-156034OB-C22), funded by MICIU/AEI/10.13039/ 501100011033 and by “ERDF/EU”. This research is also supported by the the project Cátedra de IA aplicada a la Medicina Personalizada de Precisión (Cátedras ENIA, TSI-100932-2023-3); Cátedras ENIA is funded by the Ministerio de Transformación Digital y Función Pública (Secretaría de Estado de Digitalización e Inteligencia Artificial); and by the NextGeneration EU-fund. The second and third author also thank the financial support from the Agencia Estatal de Investigación (Spain) (PID2022-137061OB-C22 funded by MICIU/AEI/10.13039/ 501100011033), the Xunta de Galicia - Conselleria de Educación, Ciencia, Universidades e Formación Profesional (Centro de investigación de Galicia acreditación 2024-2027 ED431G-2023/04 and Reference Competitive Group accreditation ED431C 2022/19) and the European Union (European Regional Development Fund - ERDF).