数字人人脸表情驱动数据

分析处理人脸图像的特征点数据得到blendshape数据，并将输出的blendshape数据应用到UE的数字人中；通过单摄像头采集人脸不同表情的图像并进行处理得到不同表情的人脸关键点，输入人脸关键点数据输出blendshape数据，根据blendshape数据实现数字人的不同的表情效果，实现驱动数字人人脸表情的效果，使其更加生动和逼真，增强数字人的表现力和交互性，使其具备更加生动、丰富、自然的表情变化能力单摄像头采集了人脸图像数据face_data，使用mediapie算法得到的人脸478个点位信息数据。将采集数据通过ai算法处理得到人脸blendshape数据，具体处理过程如下：（1）模型设计：用于进行人脸特征提取的人脸bernstein模型的定义，该模型使用了卷积神经网络（CNN）来提取特征，并包括了几个卷积层、激活函数、池化层以及全连接层。 （2）模型输入：模型的输入是一个形状为(batch_size, 3, input_length)的张量，其中batch_size表示批量大小，3表示输入通道数，input_length表示输入的序列长度（或特征点数量）。 （3）模型输出：在模型的前向传播过程中，输入首先经过一系列的卷积层、激活函数和池化层，然后通过Flatten层将特征展平成一维向量。通过一个线性层L1将特征映射到输出维度。模型的输出是一个形状为(batch_size, 52)的张量，其中52表示输出的特征维度。 将blendshape数据应用到UE数字人中，实现驱动数字人的人脸表情，增强数字人的表现力和交互性，使其具备生动、丰富、自然的表情变化能力。

Analyze and process facial landmark data from facial images to obtain blendshape data, and apply the generated blendshape data to digital humans in Unreal Engine (UE). Collect facial images of diverse expressions via a single camera, process these images to extract facial landmarks corresponding to each expression, input the facial landmark data to the model to output blendshape data, and generate different facial expression effects for digital humans based on the blendshape data, thereby driving the facial expressions of digital humans to make them more vivid and realistic, enhancing the expressiveness and interactivity of digital humans, and endowing them with more vivid, rich and natural facial expression variation capabilities.

We collected facial image data (face_data) via a single camera, and obtained 478 facial landmark information using the MediaPipe algorithm. Process the collected data through AI algorithms to generate facial blendshape data. The specific processing procedure is as follows:
1. Model Design: Define the facial Bernstein model for facial feature extraction. This model uses Convolutional Neural Networks (CNN) for feature extraction, and includes several convolutional layers, activation functions, pooling layers, and fully connected layers.
2. Model Input: The input of the model is a tensor with shape (batch_size, 3, input_length), where batch_size represents the batch size, 3 represents the number of input channels, and input_length represents the input sequence length (or the number of facial landmarks).
3. Model Output: During the forward propagation of the model, the input first passes through a series of convolutional layers, activation functions and pooling layers, then the features are flattened into a 1-dimensional vector via the Flatten layer. The features are mapped to the output dimension through a linear layer L1. The output of the model is a tensor with shape (batch_size, 52), where 52 represents the output feature dimension.

Apply the blendshape data to UE-based digital humans to drive their facial expressions, enhance the expressiveness and interactivity of digital humans, and endow them with vivid, rich and natural facial expression variation capabilities.

搜集汇总

数据集介绍

特点

该数据集包含62141条数字人人脸表情驱动数据，主要用于通过分析人脸图像特征点生成blendshape数据，以驱动数字人表情，增强其表现力和交互性。