MOSABench: Multi-Object Sentiment Analysis Benchmark for Evaluating Multimodal Large Language Models Understanding of Complex Image

Multimodal large language models (MLLMs) have shown remarkable progress in high-level semantic tasks such as visual question answering, image captioning, and emotion recognition. However, despite advancements, there remains a lack of standardized benchmarks for evaluating MLLMs performance in multi-object sentiment analysis, a key task in semantic understanding. To address this gap, we introduce MOSABench, a novel evaluation dataset designed specifically for multi-object sentiment analysis. MOSABench includes approximately 1,000 images with multiple objects, requiring MLLMs to independently assess the sentiment of each object, thereby reflecting real-world complexities. Key innovations in MOSABench include distance-based target annotation, post-processing for evaluation to standardize outputs, and an improved scoring mechanism. Our experiments reveal notable limitations in current MLLMs: while some models, like mPLUG-owl and Qwen-VL2, demonstrate effective attention to sentiment-relevant features, others exhibit scattered focus and performance declines, especially as the spatial distance between objects increases. This research underscores the need for MLLMs to enhance accuracy in complex, multi-object sentiment analysis tasks and establishes MOSABench as a foundational tool for advancing sentiment analysis capabilities in MLLMs.

多模态大语言模型（Multimodal Large Language Models，MLLMs）已在视觉问答、图像字幕生成以及情感识别等高层次语义任务中取得显著进展。然而，尽管领域内已取得诸多进展，当前仍缺乏用于评估MLLMs在多目标情感分析任务中性能的标准化基准测试集——该任务是语义理解的核心子任务之一。为填补这一空白，我们推出了MOSABench：一款专为多目标情感分析任务打造的新型评估数据集。MOSABench包含约1000张含多目标的图像，要求MLLMs独立判断每一个目标的情感倾向，以此还原真实场景的复杂性。MOSABench的核心创新点包括：基于空间距离的目标标注、用于标准化输出的评估后处理流程，以及优化后的评分机制。我们的实验揭示了当前MLLMs存在的显著局限：尽管部分模型（如mPLUG-owl与Qwen-VL2）能够有效聚焦于与情感相关的特征，但其余模型则呈现出注意力分散的问题，且性能会随着目标间空间距离的增大而下降。本研究凸显了MLLMs在复杂多目标情感分析任务中提升准确率的必要性，并确立了MOSABench作为推动MLLMs情感分析能力发展的基础工具的地位。