XD-Violence

暴力检测已在计算机视觉中研究多年。然而，以前的工作要么是肤浅的，例如短片分类和单一场景，要么供应不足，例如单一模态和基于手工制作的多模态特征。为了解决这个问题，在这项工作中，我们首先发布了一个名为 XD-Violence 的大规模多场景数据集，总时长为 217 小时，包含 4754 个带有音频信号和弱标签的未修剪视频。然后我们提出了一个包含三个并行分支的神经网络来捕获视频片段之间的不同关系并整合特征，其中整体分支使用相似性先验捕获远程依赖关系，本地化分支使用邻近先验捕获局部位置关系，并且分数分支动态捕获接近度的预测分数。此外，我们的方法还包括一个近似器来满足在线检测的需要。我们的方法在我们发布的数据集和其他现有基准上优于其他最先进的方法。此外，大量的实验结果也显示了多模态输入和建模关系的积极影响。

Violence detection has been studied in computer vision for many years. However, previous works are either superficial, such as short video classification and single-scenario tasks, or under-resourced, such as single-modal approaches or those based on handcrafted multi-modal features. To address these issues, in this work, we first release a large-scale multi-scenario dataset named XD-Violence, which has a total duration of 217 hours and contains 4754 untrimmed videos with audio signals and weak labels. We then propose a neural network with three parallel branches to capture diverse relationships between video clips and integrate features: the global branch captures long-range dependencies using a similarity prior, the localization branch captures local positional relationships using a proximity prior, and the score branch dynamically captures prediction scores related to proximity. Additionally, our method includes an approximator to meet the requirements of online detection. Our method outperforms other state-of-the-art methods on both our released dataset and other existing benchmarks. Furthermore, extensive experimental results also verify the positive impacts of multi-modal inputs and relationship modeling.

搜集汇总

背景与挑战

背景概述

XD-Violence是一个用于暴力检测的大规模多模态数据集，总时长217小时，包含4754个带有音频信号和弱标签的未修剪视频，覆盖多场景以解决以往研究中的局限性（如单一模态和场景）。该数据集由西安电子科技大学于2020年发布，旨在支持计算机视觉中多模态异常检测的研究，并促进弱监督学习方法的发展。