crag-mm-2025/crag-mm-single-turn-public

CRAG-MM数据集是一个高质量的多模态对话系统评估基准，包含关于不同复杂度级别图像的对话，用于评估AI系统的视觉理解和对话能力。数据集专注于事实性问题，提供了一个独特的图像和问答集合，以实现对可穿戴设备的全面评估。数据集包括自我中心图像和公共图像URL，涵盖13个领域。它包含四种类型的问题，从简单的问题到需要多源检索和推理的复杂问题。CRAG-MM既包含单轮对话也包含多轮对话，为多模态MM-RAG解决方案提供了全面的评估。

The CRAG-MM dataset is a high-quality conversational benchmark for multimodal assistants, featuring conversations about images with varied complexity levels, designed to evaluate AI systems visual understanding and conversational abilities. The dataset focuses on factual questions, offering a unique collection of image and question-answering sets to enable comprehensive assessment of wearable devices. It includes egocentric images captured by RayBan Meta smart glasses and public image URLs, covering 13 domains. The dataset contains four types of questions, ranging from simple ones answerable by looking at the image to complex ones requiring multi-source retrieval and reasoning. CRAG-MM encompasses both single-turn and multi-turn conversations, providing a comprehensive evaluation of MM-RAG solutions.