MemeReason

# MemeReason Extended datasets for the paper **"Adapting Reinforcement Learning with Chain-of-Thought Supervision for Explainable Detection of Hateful and Propagandistic Memes"** ([arXiv:2606.15307](https://arxiv.org/abs/2606.15307)). MemeReason augments two meme-understanding benchmarks with the supervision used to train explainable, thinking-based multimodal LLMs: - **`hateful_memes`** — the [Hateful Memes](https://ai.meta.com/tools/hatefulmemes/) benchmark (English, binary), extended with natural-language explanations, fine-grained labels (protected category and attack type, from [Mathias et al., 2021](https://github.com/facebookresearch/fine_grained_hateful_memes)), and step-by-step chain-of-thought rationales distilled from GPT-4.1. - **`armeme`** — the [ArMeme](https://huggingface.co/datasets/QCRI/ArMeme) benchmark (Arabic, 4-class propaganda detection), extended with English and Arabic explanations, fine-grained propaganda-technique annotations produced by a multi-LLM pipeline (GPT-4.1 and Llama-4-Scout annotators, consolidated by Gemini; human–LLM agreement Gwet's AC1 = 0.77), and distilled chain-of-thought rationales. > **Warning:** this dataset contains memes whose content may be disturbing or offensive. ## Fields | Field | Description | | --- | --- | | `id` | sample id from the original benchmark | | `image` | `armeme`: the meme image (embedded); `hateful_memes`: image path in the official distribution (images not redistributed; see below) | | `text` | text extracted from the meme | | `label` | `hateful` / `not-hateful`, or `propaganda` / `not-propaganda` / `not-meme` / `other` | | `explanation` | English explanation of the label (test-set explanations are human-verified) | | `explanation_ar` | Arabic explanation (`armeme` only) | | `protected_category`, `attack_type` | fine-grained hateful-meme labels (`hateful_memes` train/dev) | | `techniques` | JSON with the annotated propaganda techniques and per-technique rationales (`armeme`) | | `think` | distilled chain-of-thought trace inside `<think>` tags (train/dev only; used exclusively as training supervision) | ## Images - **ArMeme**: the meme images are embedded in the `image` column (also distributed in [QCRI/ArMeme](https://huggingface.co/datasets/QCRI/ArMeme)); `id` is the original ArMeme file path. - **Hateful Memes**: images cannot be redistributed under the dataset's license. Request them from [Meta](https://ai.meta.com/tools/hatefulmemes/); the `image` field matches the `img/*.png` layout of the official distribution. ## Usage ```python from datasets import load_dataset hateful = load_dataset("QCRI/MemeReason", "hateful_memes") armeme = load_dataset("QCRI/MemeReason", "armeme") ``` Code to reproduce the paper (training with SFT + GRPO, inference, evaluation): https://github.com/MohamedBayan/MemeReason ## Related resources - [QCRI/MemeXplain](https://huggingface.co/datasets/QCRI/MemeXplain) — the explanation-augmented benchmarks from the prior work (MemeIntel, EMNLP 2025). - [QCRI/ArMeme](https://huggingface.co/datasets/QCRI/ArMeme) — the original Arabic propagandistic memes benchmark. ## Citation If you use this dataset, please cite both papers — the WWW Companion paper that introduced the method, and the journal extension that released these data extensions: ```bibtex @inproceedings{kmainasi2026can, title = {Can Thinking Models Think to Detect Hateful Memes?}, author = {Kmainasi, Mohamed Bayan and Kutlu, Mucahid and Ezzat Shahroor, Ali and Hasnat, Abul and Alam, Firoj}, booktitle = {Companion Proceedings of the ACM Web Conference 2026}, pages = {935--944}, year = {2026} } @article{kmainasi2026memereason, title = {Adapting Reinforcement Learning with Chain-of-Thought Supervision for Explainable Detection of Hateful and Propagandistic Memes}, author = {Kmainasi, Mohamed Bayan and Kutlu, Mucahid and Shahroor, Ali Ezzat and Hasnat, Abul and Alam, Firoj}, journal = {arXiv preprint arXiv:2606.15307}, year = {2026} } ``` ## License CC BY-NC-SA 4.0. The `hateful_memes` extension is released for research use; the underlying images remain subject to the Hateful Memes dataset license. The `armeme` extension follows the ArMeme license (CC BY-NC-SA 4.0).