Mantis-Instruct 多图像指令调优数据集

Mantis-Instruct 数据集是一个由滑铁卢大学、清华大学和上海人工智能实验室于 2024 年发布的，相关论文成果为「MANTIS: Interleaved Multi-Image Instruction Tuning」。专注于多图像指令调优的文本-图像交错多模态数据集，由 14 个子集组成，包含 721K 个例子，用于训练 Mantis 模型系列。该数据集覆盖了多种多图像技能，包括共指、推理、比较和时间理解等多个方面。它不仅包括来自现有数据集的 10 个子集，还新增了 4 个新策划的数据集，旨在扩展模型的推理技能和其他多图像技能。

The Mantis-Instruct dataset was released in 2024 by the University of Waterloo, Tsinghua University, and the Shanghai AI Laboratory, with its associated research paper titled "MANTIS: Interleaved Multi-Image Instruction Tuning". It is a text-image interleaved multimodal dataset focused on multi-image instruction tuning, consisting of 14 subsets and 721K examples, designed for training the Mantis model family. This dataset covers a variety of multi-image skills including coreference resolution, reasoning, comparison, temporal understanding, and more. It not only includes 10 subsets sourced from existing datasets but also adds 4 newly curated datasets, aiming to expand the model's reasoning skills and other multi-image capabilities.

搜集汇总

背景与挑战

背景概述

Mantis-Instruct是一个由滑铁卢大学、清华大学和上海人工智能实验室于2024年发布的多图像指令调优数据集，包含721K个例子和14个子集，旨在训练模型处理共指、推理、比较和时间理解等多图像技能。该数据集整合了现有资源并新增了四个策划子集，以扩展模型的推理能力。