DataSheet1_Study on Automatic 3D Facial Caricaturization: From Rules to Deep Learning.PDF

Facial caricature is the art of drawing faces in an exaggerated way to convey emotions such as humor or sarcasm. Automatic caricaturization has been explored both in the 2D and 3D domain. In this paper, we propose two novel approaches to automatically caricaturize input facial scans, filling gaps in the literature in terms of user-control, caricature style transfer, and exploring the use of deep learning for 3D mesh caricaturization. The first approach is a gradient-based differential deformation approach with data driven stylization. It is a combination of two deformation processes: facial curvature and proportions exaggeration. The second approach is a GAN for unpaired face-scan-to-3D-caricature translation. We leverage existing facial and caricature datasets, along with recent domain-to-domain translation methods and 3D convolutional operators, to learn to caricaturize 3D facial scans in an unsupervised way. To evaluate and compare these two novel approaches with the state of the art, we conducted the first user study of facial mesh caricaturization techniques, with 49 participants. It highlights the subjectivity of the caricature perception and the complementarity of the methods. Finally, we provide insights for automatically generating caricaturized 3D facial mesh.

人脸夸张漫画（Facial Caricature）是一种以夸张手法绘制人脸，以传递幽默、讽刺等情感的艺术形式。自动夸张漫画化技术已在二维（2D）与三维（3D）领域均得到探索。本文提出两种可对输入人脸扫描模型进行自动夸张漫画化的全新方法，填补了现有研究在用户可控性、漫画风格迁移以及探索深度学习用于三维网格（3D Mesh）夸张漫画化等方向的空白。第一种方法为基于梯度的微分形变方法，结合数据驱动的风格化处理，融合了两类形变流程：人脸曲率夸张与比例夸张。第二种方法是用于非配对人脸扫描至三维夸张漫画转换的生成对抗网络（GAN, Generative Adversarial Network）。本研究利用现有面部与漫画数据集，结合最新的域间迁移方法与三维卷积算子，以无监督方式实现三维人脸扫描的夸张漫画化。为评估并对比这两种全新方法与当前前沿技术，我们开展了首次针对三维网格人脸夸张漫画化技术的用户研究，共招募49名参与者。该研究凸显了夸张漫画感知的主观性，以及两种方法的互补性。最后，本文为自动生成夸张漫画化三维人脸网格提供了参考思路。