Deep features for assessing smile genuineness from video sequences

This package contains the deep features extracted from the UvA-NEMO database (https://www.uva-nemo.org/) accompanied with a script (read_bin_file.py) that can be conveniently used for reading these features from the binary files. To extract features from the cropped faces (color images of size 224x224) within each video frame, we exploited the feature extraction part of a VGG-19 convolutional neural network pre-trained on ImageNet (note that the UvA-NEMO images are thus normalized using the mean and standard deviation of ImageNet). For each video sequence, we build a single feature vector with the features extracted from the face regions in all frames, appended one after another. We dump the features from the final layer in the feature extraction part of VGG-19 (after the adaptive average pooling). For each video sequence, we store (in the following order): the number of features, the number of frames, and the height and width of the adaptive average pooling (used after the final convolutional layer), and the features (extracted from the first frame presenting a face, followed by the features extracted from the second frame presenting a face, and so forth). Overall, we have 1240 binary files including the features extracted for 1240 UvA-NEMO sequences manifesting posed and genuine smiles.

本数据包包含从UvA-NEMO数据库（https://www.uva-nemo.org/）中提取的深度特征，附带一个可便捷读取该类二进制特征文件的脚本read_bin_file.py。为从每帧视频中的裁剪人脸（尺寸为224×224的彩色图像）提取特征，我们采用了在ImageNet数据集上预训练的VGG-19卷积神经网络的特征提取分支。需注意，UvA-NEMO数据集的图像将按照ImageNet的均值与标准差进行归一化处理。针对每个视频序列，我们将所有帧中人脸区域提取得到的特征依次拼接，得到单个特征向量。我们导出的特征取自VGG-19特征提取分支的最后一层，即自适应平均池化操作之后的输出特征。针对每个视频序列，我们按以下顺序存储数据：特征总数、视频帧数、最终卷积层后所使用的自适应平均池化的高与宽，以及特征序列（按帧顺序依次为首张含人脸帧的特征、第二张含人脸帧的特征，以此类推）。总体而言，本数据集共包含1240个二进制特征文件，对应1240个UvA-NEMO视频序列，这些序列涵盖了摆姿微笑与真实微笑两类样本。