FBK-MT/MCIF

MCIF（多模态跨语言指令遵循）是一个基于科学讲座的多语言人工标注基准测试数据集，旨在评估跨语言和多模态环境下的指令遵循能力，涵盖长格式和短格式输入。MCIF包含语音、视觉和文本三种核心模态，支持英语、德语、意大利语和中文四种语言，能够全面评估多语言大模型（MLLMs）在不同语言和模态上下文信息中的指令解释能力。数据集分为长格式和短格式输入，以及固定提示和混合提示两种提示类型，适用于自动语音识别、问答、摘要、视觉问答和翻译等多种任务。

MCIF (Multimodal Crosslingual Instruction Following) is a multilingual human-annotated benchmark based on scientific talks that is designed to evaluate instruction-following in crosslingual, multimodal settings over both short- and long-form inputs. MCIF spans three core modalities -- speech, vision, and text -- and four diverse languages (English, German, Italian, and Chinese), enabling a comprehensive evaluation of MLLMs abilities to interpret instructions across languages and combine them with multimodal contextual information. The dataset is organized into long and short tracks with fixed and mixed prompt types, supporting tasks such as automatic speech recognition, question answering, summarization, visual question answering, and translation.