Large-scale datasets for facial tampering detection with inpainting techniques

DeepFake technology， born with the continuous maturation of deep learning techniques， primarily utilizes neural networks to create non-realistic faces. This method has enriched people’s lives as computer vision advances and deep learning technologies mature. It has revolutionized the film industry by generating astonishing visuals and reducing production costs. Similarly， in the gaming industry， it has facilitated the creation of smooth and realistic animation effects. However， the malicious use of image manipulation to spread false information poses significant risks to society， casting doubt on the authenticity of digital content in visual media. Forgery techniques encompass four main categories： face reenactment， face replacement， face editing， and face synthesis. Face editing， a commonly employed image manipulation method， involves falsifying facial features by modifying the information related to the five facial regions. As one of the commonly employed methods in facial editing， image inpainting technology involves utilizing known content from an image to fill in missing areas， aiming to restore the image in a way that aligns as closely as possible with human perception. In the context of facial forgery， image inpainting is primarily used for identity falsification， wherein facial features are altered to achieve the goal of replacing a face. The use of image inpainting for facial manipulation similarly introduces significant disruption to people’s lives. To support research on detection methods for such manipulations， this paper produced a large-scale dataset for face manipulation detection based on inpainting techniques.This paper specifically focuses on the field of image tampering detection， utilizing two classic datasets： the high-quality CelebA-HQ dataset， comprising 25 000 high-resolution （1 024 × 1 024 pixels） celebrity face images， and the low-quality FF++ dataset， consisting of 15 000 face images extracted from video frames. On the basis of the two datasets， facial feature regions （eyebrows， eyes， nose， mouth， and the entire facial area） are segmented using image segmentation methods. Corresponding mask images are created， and the segmented facial regions are directly obscured on the original image. Two deep neural network-based inpainting methods （image inpainting via conditional texture and structure dual generation （CTSDG） and recurrent feature reasoning for image inpainting （RFR）） along with a traditional inpainting method （struct completion（SC）） were employed. The deep neural network methods require the provision of mask images to indicate the areas for inpainting， while the traditional method could directly perform inpainting on segmented facial feature images. The facial regions were inpainted using these three methods， resulting in a large-scale dataset comprising 600 000 images. This extensive dataset incorporates diverse pre-processing techniques， various inpainting methods， and includes images with different qualities and inpainted facial regions. It serves as a valuable resource for training and testing in related detection tasks， offering a rich dataset for subsequent research in the field， and also establishes a meaningful benchmark dataset for future studies in the domain of face tampering detection.