WildDeepfake

近年来，一种名为 deepfake 的人脸交换技术 Deepfake 的滥用引起了公众的极大关注。 Deepfake 操纵深度学习技术将视频中的一个人的脸替换为另一个人的脸，而不会留下明显的痕迹。到目前为止，已经制作了大量的 deepfake 视频（也称为“deepfakes”）并上传到互联网，这需要制定有效的对策。针对 deepfake 的一个有希望的对策是 deepfake 检测。已经发布了几个 deepfake 数据集来支持 deepfake 检测器的训练和测试，例如 DeepfakeDetection 和 FaceForensics++。虽然这极大地推进了 deepfake 检测，但这些数据集中的大部分真实视频都是由少数志愿者在有限的场景中拍摄的，并且假视频是由研究人员使用一些流行的 deepfake 软件制作的。在这些数据集上开发的检测器在用于检测野外大量的 deepfake 视频（上传到各种视频共享网站的视频）时可能会失效。为了更好地支持检测真实世界的 deepfake，在本文中，我们引入了一个新的数据集 WildDeepfake，该数据集包含从完全从互联网上收集的 707 个 deepfake 视频中提取的 7,314 个面部序列。 WildDeepfake 是一个小型数据集，除了现有数据集外，还可用于开发更有效的检测器来对抗真实世界的 deepfake。我们对现有的和我们的 WildDeepfake 数据集上的一组基线检测网络进行了系统评估，并表明WildDeepfake 确实是一个更具挑战性的数据集，其中检测性能可能会急剧下降。我们还提出了两个（例如 2D 和 3D）基于注意力的 Deepfake 检测网络（ADDNets），以利用真/假人脸上的注意力掩码来改进检测。我们凭经验验证了 ADDNets 在现有和 WildDeepfake 上的有效性。

In recent years, the abuse of deepfake, a face-swapping technology, has aroused great public concern. Deepfake manipulates deep learning techniques to replace one person's face in a video with another's without leaving noticeable traces. So far, a large number of deepfake videos (also known as "deepfakes") have been created and uploaded to the Internet, which calls for effective countermeasures. A promising countermeasure against deepfakes is deepfake detection. Several deepfake datasets have been released to support the training and testing of deepfake detectors, such as DeepfakeDetection and FaceForensics++. While this has greatly advanced deepfake detection, most of the real videos in these datasets were shot by a small number of volunteers in limited scenarios, and the fake videos were created by researchers using some popular deepfake software. Detectors developed on these datasets may fail when used to detect a large number of deepfake videos in the wild (i.e., videos uploaded to various video-sharing websites). To better support the detection of real-world deepfakes, we introduce a new dataset, WildDeepfake, in this paper. This dataset contains 7,314 face sequences extracted from 707 deepfake videos completely collected from the Internet. WildDeepfake is a small-scale dataset that can be used in addition to existing datasets to develop more effective detectors to combat real-world deepfakes. We conducted a systematic evaluation of a set of baseline detection networks on both existing datasets and our WildDeepfake dataset, and demonstrated that WildDeepfake is indeed a more challenging dataset, where detection performance can drop sharply. We also propose two attention-based Deepfake detection networks (ADDNets, e.g., 2D and 3D variants) to leverage attention masks on real/fake faces for improved detection. We empirically verified the effectiveness of ADDNets on both existing datasets and WildDeepfake.

搜集汇总

背景与挑战

背景概述

WildDeepfake是一个从互联网收集的deepfake视频数据集，包含707个视频和7,314个面部序列，旨在支持开发更有效的真实世界deepfake检测器。与现有数据集相比，它更具挑战性，检测性能可能显著下降。