A Multimodal Dataset for Multiple-Choice Question Difficulty Estimation

Multiple-choice questions (MCQs) are central to educational assessment, where item difficulty plays a key role in test reliability and adaptivity. This article introduces a multimodal dataset of 11,199 MCQs that integrates textual and visual content to support content-based difficulty estimation. Each MCQ consists of a question, four answer options, and an associated image accompanied by an expert-authored caption that describes only the observable visual content, independent of the question or answer options. The dataset includes three forms of expert annotation: (i) visual captions focused exclusively on image content, (ii) categorical difficulty labels assigned using a standardized rubric that considers the overall cognitive effort required to solve the item given its textual and visual components, and (iii) Bloom’s taxonomy–aligned cognitive labels reflecting the intended learning objective of each question. For each image, three MCQs are authored to capture graded difficulty variation while preserving a consistent visual context, spanning topics in the natural and social sciences. Trained secondary-school educators contribute to annotation, verification, and disagreement reconciliation through a structured, multi-phase workflow, ensuring pedagogically grounded and internally consistent labels. Baseline validation checks confirm data integrity and reproducibility. The dataset contains no personal or sensitive information and is released publicly to support research in multimodal reasoning, educational measurement, and intelligent tutoring systems.

This dataset has multiple varieties in it

(a) A multimodal dataset with expert annotated difficulty labels for difficulty estimation.

(b) Also additional subset with noisy annotation of labels and questions for visual modality checking, used for testing model hallucination and other associated tasks.

License: This dataset contains two components with different licenses. The images are sourced from the ScienceQA dataset and are licensed under Creative Commons Attribution–NonCommercial–ShareAlike 4.0 (CC-BY-NC-SA 4.0). All newly created content in this release-including image captions, multiple-choice questions, answer options, Bloom’s-taxonomy labels, difficulty labels, and all associated metadata-is licensed under Creative Commons Attribution–ShareAlike 4.0 (CC-BY-SA 4.0). Any redistribution or reuse that includes the original images must comply with the CC-BY-NC-SA 4.0 terms. The annotation files (questions, captions, labels, and metadata) may be used independently under the more permissive CC-BY-SA 4.0 license, enabling reuse in commercial and non-commercial settings when images are not redistributed.

Note: The page is under update, we will release latest version soon.

For immediate queries contact [e-mail]

数据集详情

多项选择题（Multiple-Choice Questions, MCQs）是教育测评的核心题型，题目难度对测试的信度与自适应特性至关重要。本文介绍了一个包含11199道多项选择题的多模态数据集，该数据集整合文本与视觉内容，可支撑基于内容的难度预估研究。每道多项选择题包含题干、四个选项，以及一张配套图片，且附带由专家撰写的图像说明文字，该说明仅描述可观测的视觉内容，与题干或选项无关。本数据集包含三类专家标注：（1）仅针对图像内容的视觉说明标注；（2）采用标准化评分细则生成的分类难度标签，该细则会结合题目文本与视觉组件，考量解答该题所需的整体认知负荷；（3）与布鲁姆认知层级分类（Bloom’s Taxonomy）对齐的认知标签，用于体现每道题预设的学习目标。针对每张图片，我们编写了三道多项选择题，以在保持一致视觉背景的前提下体现难度梯度变化，题目涵盖自然科学与社会科学领域。经专业培训的中学教育工作者通过结构化多阶段流程参与标注、校验与分歧协调，确保标注符合教育学规范且内部一致。基线验证检查可确保数据完整性与可复现性。本数据集不包含任何个人或敏感信息，已公开发布，可用于多模态推理、教育测评以及智能导学系统（intelligent tutoring systems）相关研究。

我们提供该数据集的两个版本：
适用于多模态难度预估的通用版本，以及
专为多模态视角下的难度预估研究设计的模态聚焦版本，尤其适用于视觉语言模型（Vision-Language Models, VLMs）的相关评测。

如需获取最终数据集，请发送请求详情至邮箱[e-mail]。

许可证说明
本数据集包含两类遵循不同许可协议的组件。其中图像源自ScienceQA数据集，采用知识共享署名-非商业性使用-相同方式共享4.0国际许可协议（CC-BY-NC-SA 4.0）进行授权。本发布版本中所有新创建的内容——包括图像说明文字、多项选择题、选项、布鲁姆认知标签、难度标签及元数据——均采用知识共享署名-相同方式共享4.0国际许可协议（CC-BY-SA 4.0）授权。任何包含该图像的再分发或复用行为必须符合CC-BY-NC-SA 4.0的条款要求，而标注文件可单独在CC-BY-SA 4.0协议下使用。

版本说明
本记录取代所有过往发布版本，用户应引用并使用本记录关联的当前版本数据集。