MOSABench: Multi-Object Sentiment Analysis Benchmark for Evaluating Multimodal Large Language Models Understanding of Complex Image

Multimodal large language models (MLLMs) have shown remarkable progress in high-level semantic tasks such as visual question answering, image captioning, and emotion recognition. However, despite advancements, there remains a lack of standardized benchmarks for evaluating MLLMs performance in multi-object sentiment analysis, a key task in semantic understanding. To address this gap, we introduce MOSABench, a novel evaluation dataset designed specifically for multi-object sentiment analysis. MOSABench includes approximately 1,000 images with multiple objects, requiring MLLMs to independently assess the sentiment of each object, thereby reflecting real-world complexities. Key innovations in MOSABench include distance-based target annotation, post-processing for evaluation to standardize outputs, and an improved scoring mechanism. Our experiments reveal notable limitations in current MLLMs: while some models, like mPLUG-owl and Qwen-VL2, demonstrate effective attention to sentiment-relevant features, others exhibit scattered focus and performance declines, especially as the spatial distance between objects increases. This research underscores the need for MLLMs to enhance accuracy in complex, multi-object sentiment analysis tasks and establishes MOSABench as a foundational tool for advancing sentiment analysis capabilities in MLLMs.

多模态大语言模型（MLLMs）在视觉问答、图像描述和情感识别等高级语义任务方面取得了显著的进展。然而，尽管取得了进步，但在多对象情感分析领域，即语义理解中的关键任务，仍然缺乏评估MLLMs性能的标准基准。为填补这一空白，我们提出了MOSABench，这是一个专为多对象情感分析设计的创新评估数据集。MOSABench包含约1,000张包含多个对象的图像，要求MLLMs独立评估每个对象的情感，从而反映现实世界的复杂性。MOSABench的关键创新包括基于距离的目标标注、用于标准化输出的后处理以及改进的评分机制。我们的实验揭示了当前MLLMs的显著局限性：虽然一些模型，如mPLUG-owl和Qwen-VL2，能够有效关注与情感相关的特征，但其他模型则表现出分散的注意力，性能随对象间空间距离的增加而下降。本研究强调了MLLMs在复杂的多对象情感分析任务中提高准确性的必要性，并将MOSABench确立为提升MLLMs情感分析能力的基础工具。