LOKI

随着人工智能生成内容的快速发展，未来的互联网可能会被合成数据淹没，使得真实可信的多模态数据的辨别变得越来越具有挑战性。合成数据检测因此引起了广泛关注，大型多模态模型（LMM）在此任务中的表现引起了广泛兴趣。LMM 可以 为其真实性判断 提供自然语言解释，增强了合成内容检测的可解释性。同时，区分真实数据和合成数据的任务有效地考验了 LMM 的感知、知识和推理能力。为此，我们推出了LOKI ，这是一个新颖的基准，旨在评估 LMM 检测多种模态合成数据的能力。LOKI涵盖视频、 图像、3D、文本和音频模态，包含 26 个子类别中 18,000 个精心策划的问题，具有明确的难度级别。该基准包括粗粒度判断和多项选择题，以及 细粒度异常选择和解释任务，从而可以对 LMM 进行全面分析。我们在LOKI上评估了 22 个开源 LMM 和 6 个闭源模型，突出了它们作为合成数据检测器的潜力，同时也揭示了 LMM 功能开发中的一些局限性。With the rapid development of AI-generated content, the future Internet may be flooded with synthetic data, making the discrimination of authentic and trustworthy multimodal data increasingly challenging. Synthetic data detection has thus attracted widespread attention, and the performance of Large Multimodal Models (LMMs) on this task has also garnered significant interest. LMMs can provide natural language explanations for their authenticity judgments, enhancing the interpretability of synthetic content detection. Meanwhile, the task of distinguishing real and synthetic data effectively tests the perception, knowledge, and reasoning capabilities of LMMs. To this end, we introduce LOKI, a novel benchmark designed to evaluate the ability of LMMs to detect multi-modal synthetic data. LOKI covers video, image, 3D, text, and audio modalities, and contains 18,000 carefully curated questions across 26 subcategories with explicit difficulty levels. This benchmark includes coarse-grained judgment and multiple-choice questions, as well as fine-grained anomaly selection and explanation tasks, enabling comprehensive analysis of LMMs. We evaluated 22 open-source LMMs and 6 closed-source models on LOKI, highlighting their potential as synthetic data detectors while also revealing some limitations in LMM functional development.

搜集汇总

背景与挑战

背景概述

LOKI是一个多模态合成数据检测基准，专门设计用于全面评估大型多模态模型（LMM）在检测视频、图像、3D、文本和音频等合成数据方面的能力。该数据集包含26个子类别和超过18,000个问题，涵盖判断、多项选择和异常解释等多种任务类型，以测试LMM的感知和推理能力。它发布于2024年，旨在推动合成数据检测研究，并已用于评估多个开源和闭源模型。