WhatsApp Vaccine Discourse (WhaVax): An Expert-Annotated Dataset and Benchmark for Health Misinformation Detection

A new expert-annotated dataset of vaccine-related WhatsApp messages collected from large Brazilian public groups across multiple pandemic years. The dataset is built through careful keyword filtering, semantic deduplication, and a multi-stage annotation protocol with medical specialists, resulting in a reliable gold-standard corpus with substantial agreement. Files: WhaVax_dataset.csv: Annotated WhatsApp vaccine misinformation dataset in csv context_llm_training.py: Evaluation of large language models via In-Context Learningslm_models_training.py: Training and evaluation of small language modelsclassic_models_training.py: Training of classical machine learning classifiersgeneral_analysis.ipynb: Exploratory data analysis and annotation statisticsgeneral_plots.ipynb: Visualization of dataset characteristics and results WhaVax_dataset.csv columns: Message: Message textgroup: Data related to the group in which the message was sentsender: ID of the person who sent the messageav1_desinfo, av2_desinfo, av3_desinfo, av4_desinfo: Labeling of each of the annotatorsis_quote: Whether this message is a reply to another messageddd_code: Area code of the person who sent the messagecountry_code: Country code of the person who sent the messagedate: Date and time the message was sentmessage_id: Message IDforwarded: Message forwarding level If you use this dataset, we appreciate it if you cite the following paper: dos Santos, J. H., C. S. Reis, J., Melo, P. de F., Hecksher Olivetti, J. F., Silva, T. H., Guimaraes, M. G., … Lima, C. X. (2026). WhatsApp Vaccine Discourse (WhaVax): An Expert-Annotated Dataset and Benchmark for Health Misinformation Detection. Proceedings of the International AAAI Conference on Web and Social Media, 20(1), 2768–2779. https://doi.org/10.1609/icwsm.v20i1.42781