MMRel

MMRel是由南洋理工大学和阿里巴巴达摩院合作开发的大型多模态关系理解数据集，包含超过15,000个问题-答案对，涵盖三个不同的领域，确保了数据集的大规模和多样性。数据集内容包括空间、动作和比较三种关系类别，数据来源于真实图像、SDXL和Dall-E生成的合成图像。创建过程中采用了半自动数据收集管道，利用MLLMs生成图像和注释，并通过人工审核确保数据质量。MMRel主要用于评估和提升MLLMs在关系理解方面的能力，特别是在视觉-语言任务中的应用。MMRel is a large-scale multimodal relational understanding dataset co-developed by Nanyang Technological University and Alibaba DAMO Academy. It contains over 15,000 question-answer pairs and covers three distinct domains, ensuring the dataset's large scale and diversity. The dataset includes three relational categories: spatial, action, and comparative relationships. Its data originates from real images as well as synthetic images generated by SDXL and DALL-E. During its development, a semi-automatic data collection pipeline was employed, leveraging MLLMs to generate images and annotations, with manual reviews carried out to guarantee data quality. MMRel is primarily intended to evaluate and improve the relational understanding capabilities of MLLMs, particularly in vision-language tasks.

搜集汇总

背景与挑战

背景概述

MMRel是一个由南洋理工大学和阿里巴巴达摩院合作构建的大型多模态关系理解数据集，包含超过15,000个问题-答案对，涵盖空间、动作和比较三种关系类别，数据来源于真实图像和合成图像。它通过半自动管道生成并人工审核，旨在评估和提升多模态大语言模型在视觉-语言任务中的关系理解能力。