MAmmoTH-VL-Instruct-12M

# MAmmoTH-VL-Instruct-12M [🏠 Homepage](https://mammoth-vl.github.io/) | [🤖 MAmmoTH-VL-8B](https://huggingface.co/MAmmoTH-VL/MAmmoTH-VL-8B) | [💻 Code](https://github.com/MAmmoTH-VL/MAmmoTH-VL) | [📄 Arxiv](https://arxiv.org/abs/2412.05237) | [📕 PDF](https://arxiv.org/pdf/2412.05237) | [🖥️ Demo](https://huggingface.co/spaces/paralym/MAmmoTH-VL-8B) ## Introduction Our simple yet scalable visual instruction data rewriting pipeline consists of three steps: manual data source collection, rewriting using MLLMs/LLMs, and filtering via the same MLLM as a judge. Examples below illustrate transformations in math and science categories, showcasing detailed, step-by-step responses.  ## The data distribution of MAmmoTH-VL-Instruct (12M)  ## Citation ``` @article{guo2024mammothvlelicitingmultimodalreasoning, title={MAmmoTH-VL: Eliciting Multimodal Reasoning with Instruction Tuning at Scale}, author={Jarvis Guo and Tuney Zheng and Yuelin Bai and Bo Li and Yubo Wang and King Zhu and Yizhi Li and Graham Neubig and Wenhu Chen and Xiang Yue}, year={2024}, eprint={2412.05237}, archivePrefix={arXiv}, primaryClass={cs.CL}, url={https://arxiv.org/abs/2412.05237}, } ```# MAmmoTH-VL-Instruct-12M [🏠 主页](https://mammoth-vl.github.io/) | [🤖 MAmmoTH-VL-8B](https://huggingface.co/MAmmoTH-VL/MAmmoTH-VL-8B) | [💻 代码](https://github.com/MAmmoTH-VL/MAmmoTH-VL) | [📄 arXiv](https://arxiv.org/abs/2412.05237) | [📕 全文PDF](https://arxiv.org/pdf/2412.05237) | [🖥️ 演示](https://huggingface.co/spaces/paralym/MAmmoTH-VL-8B) ## 简介 我们这款简洁却具备可扩展性的视觉指令数据重写流水线包含三个核心步骤：手动采集数据源、借助多模态大语言模型（Multimodal Large Language Models, MLLMs）/大语言模型（Large Language Models, LLMs）进行数据重写，以及以同款多模态大语言模型作为评判器完成筛选。下述示例展示了数学与科学类别下的数据变换过程，并呈现了详尽的分步响应内容。  ## MAmmoTH-VL-Instruct（12M）的数据分布  ## 引用 @article{guo2024mammothvlelicitingmultimodalreasoning, title={MAmmoTH-VL: Eliciting Multimodal Reasoning with Instruction Tuning at Scale}, author={Jarvis Guo and Tuney Zheng and Yuelin Bai and Bo Li and Yubo Wang and King Zhu and Yizhi Li and Graham Neubig and Wenhu Chen and Xiang Yue}, year={2024}, eprint={2412.05237}, archivePrefix={arXiv}, primaryClass={cs.CL}, url={https://arxiv.org/abs/2412.05237}, }

搜集汇总

背景与挑战

背景概述

MAmmoTH-VL-Instruct-12M是一个包含1200万样本的视觉指令数据集，通过手动收集数据源、使用MLLMs/LLMs重写，并经过MLLM过滤的流程生成，旨在通过指令调优引发多模态推理能力。