11kHands人手拍照数据集，大型手部图像数据集的性别识别和生物统计识别

欢迎来到11k手数据集，这是一个由190名年龄在18-75岁之间的受试者的11076张手部图像（1600 x 1200像素）组成的集合。每个受试者都被要求打开和关闭他的左右手的手指。每只手都从背侧和掌侧进行拍摄，背景为统一的白色，并与相机保持大致相同的距离。每张图片都有一个与之相关的元数据记录，其中包括 (1)主体ID，(2)性别，(3)年龄，(4)肤色，以及(5)一组被拍摄的手的信息，即右手或左手，手的侧面（背侧或掌侧），以及指手部图像是否包含饰品、指甲油或不规则的逻辑指标。拟议的数据集有大量的手部图像，有更详细的元数据。该数据集对于合理的学术公平使用是免费的。 The following Figures show the basic statistics of the proposed dataset. The first Figure contains the following: Top: the distribution of skin colors in the dataset is shown. The number of images for each skin color category is written in the top right of the figure. The skin detection process was performed using the skin detection algorithm proposed by by Conaire et al. [1]. Bottom: shows the statistics of 1) the number of subjects, 2) hand images (dorsal- and palmar- sides), 3) hand images with accessories, and 4) hand images with nail polish. The second Figure shows the age distribution of the subjects and images of the proposed dataset. [1] Conaire, C. O., O'Connor, N. E., & Smeaton, A. F.. Detector adaptation by maximising agreement between independent data sources. In CVPR'07 [1] Sun, Z., Tan, T., Wang, Y., & Li, S. Z. (2005, June). Ordinal palmprint represention for personal identification. In Computer Vision and Pattern Recognition, 2005. CVPR 2005. IEEE Computer Society Conference on (Vol. 1, pp. 279-284). IEEE. [2] Yoruk, E., Konukoglu, E., Sankur, B., & Darbon, J. (2006). Shape-based hand recognition. IEEE transactions on image processing, 15(7), 1803-1815. [3] Yörük, E., Dutağaci, H., & Sankur, B. (2006). Hand biometrics. Image and Vision Computing, 24(5), 483-497. [4] Hu, R. X., Jia, W., Zhang, D., Gui, J., & Song, L. T. (2012). Hand shape recognition based on coherent distance shape contexts. Pattern Recognition, 45(9), 3348-3359. [5] Kumar, A. (2008, December). Incorporating cohort information for reliable palmprint authentication. In Computer Vision, Graphics & Image Processing, 2008. ICVGIP'08. Sixth Indian Conference on(pp. 583-590). IEEE. [6] Ferrer, M. A., Morales, A., Travieso, C. M., & Alonso, J. B. (2007, October). Low cost multimodal biometric identification system based on hand geometry, palm and finger print texture. In Security Technology, 2007 41st Annual IEEE International Carnahan Conference on (pp. 52-58). IEEE. We present a two-stream CNN for gender classification using the proposed dataset. We then employ this trained two-stream CNN as a feature extractor for both gender classification and biometric identification. The latter is handled using two different approaches. In the first approach, we construct a feature vector from the deep features, extracted from the trained CNN, to train a support vector machine (SVM) classifier. In the second approach, three SVM classifiers are fed by the deep features extracted from different layers of the trained CNN and one SVM classifier is trained using the local binary pattern (LBP) features in order to improve the correct identification rate obtained by summing up the classification scores of all SVM classifiers. You can download the trained models and classifiers from tables below. As we have a bias towards the number of female hand images (see the statistics above), we use 1,000 dorsal hand images of each gender for training and 500 dorsal hand images of each gender for testing. The images are picked randomly such that the training and testing sets are disjoint sets of subjects, meaning if the subject's hand images appear in the training data, this subject is excluded from the testing data and vice-versa. The same is done for palmar side hand images. For each side, we repeat the experiment 10 times to avoid overfitting problems and consider the average of accuracy as the evaluation metric. For comparisons, we have train different image classification methods using the 10 sets of training and testing pairs. The methods are: (1) bag of visual words (BoW), (2) fisher vector, (3) Alexnet (CNN), (4) VGG-16 (CNN), (5) VGG-19 (CNN), and (6) Googlenet (CNN). For the first image classification frameworks (BoW and FV), we have used three different feature descriptors, which are: (1) SIFT, (2) C-SIFT, and (3) rgSIFT. For further comparisons, we recommend to use the same evaluation criterion. To download the 10 sets of training and testing pairs that have been used in our experiments, see the following Table: Each set contains the following files: You can use this Matlab code to extract the images used in each experiments. The code generates 10 directories, each one contains the training and testing sets for each gender. Then you can use the imageDatastore function to load them (see CNN_training.m source code). Trained CNN models, SVM classifiers, and results If the Matlab Neural Network Toolbox Model for Network support package is not installed, then the function provides a link to the required support package in the Add-On Explorer. To install the support package, click the link, and then click Install. Check that the installation is successful by typing the model name (e.g. alexnet, vgg16, vgg19, and googlenet) at the command line. *requires Matlab 2016 or higher. **requires Matlab 2017b or higher. +trained SVM classifiers using our CNN model as a feature extractor, as described in the paper. The SVM classifiers were trained using the concatenated feature vector in which features from fc9 of the 1st stream, fc10 of the 2nd stream and the fusion fully connected layer are concatenated into one vector. The LBP/SVM classifiers were trained using the concatenated feature vector in which LBP features and features from fc9 of the 1st stream, fc10 of the 2nd stream and the fusion fully connected layer are concatenated into one vector. For biometric identification, we work with different training and testing sets. We use 10 hand images for training and 4 hand images for testing of each hand side (palmar or dorsal) of 80, 100, and 120 subjects. We repeat the experiment 10 times, with the subjects and images picked randomly each time. We adopt the average identification accuracy as the evaluation metric. For further comparisons, we recommend to use the same evaluation criterion. To download the 10 sets of training and testing pairs that have been used in our experiments, see the following Table: Each set contains the following files: You can use this Matlab code to extract the images used in each experiments. The code generates 10 directories, each one contains the training and testing sets for each set of subjects. Each filename contains the ID of the subject. For example, 0000000_Hand_0000055.jpg means this image for subject number 0000000, the rest of the file name is the original image name. You can use this Matlab code to load all image filenames and extract the corresponding IDs. Trained SVM Classifiers and results *trained SVM classifiers using our CNN model as a feature extractor, as described in the paper. Each .mat file contains a Classifier object where: Questions and comments can be sent to: mafifi[at]eecs[dot]yorku[dot]ca or m.afifi[at]aun[dot]edu[dot]eg

欢迎来到11k手部数据集(11k Hand Dataset)，该数据集由190名年龄介于18至75岁之间的受试者的11076张手部图像（分辨率为1600×1200像素）构成。每位受试者均被要求完成左右手手指的开合动作。每只手均分别从背侧与掌侧两个视角进行拍摄，背景采用统一的纯白色，且手部与相机的距离保持大致一致。每张图像均配有对应的元数据记录，包含以下信息：(1) 受试者ID；(2) 性别；(3) 年龄；(4) 肤色；(5) 被拍摄手部的相关信息，即左手或右手、手部拍摄视角（背侧或掌侧），以及表征该手部图像是否包含饰品、指甲油或存在畸形的逻辑标记。本数据集包含海量手部图像与详尽的元数据，可免费用于合理的学术使用。

以下图表展示了本数据集的基本统计信息。第一张图表包含如下内容：上方为数据集中的肤色分布情况，各肤色类别对应的图像数量标注于图表右上角；肤色检测流程采用Conaire等人提出的肤色检测算法实现[1]。下方则展示了四项统计数据：1) 受试者数量；2) 背侧与掌侧手部图像的数量；3) 带有饰品的手部图像数量；4) 带有指甲油的手部图像数量。第二张图表展示了本数据集的受试者年龄分布与图像分布情况。

[1] Conaire, C. O., O'Connor, N. E., & Smeaton, A. F. 通过最大化独立数据源间的一致性实现检测器适配[C]//2007年计算机视觉与模式识别会议（CVPR'07）.
[2] Sun, Z., Tan, T., Wang, Y., & Li, S. Z. 用于个人身份识别的序数掌纹表示[C]//2005年计算机视觉与模式识别会议（CVPR 2005）. IEEE计算机学会会议论文集（第1卷）, 2005: 279-284.
[3] Yoruk, E., Konukoglu, E., Sankur, B., & Darbon, J. 基于形状的手部识别[J]. IEEE图像处理汇刊, 2006, 15(7): 1803-1815.
[4] Yörük, E., Dutağaci, H., & Sankur, B. 手部生物特征识别[J]. 图像与视觉计算, 2006, 24(5): 483-497.
[5] Hu, R. X., Jia, W., Zhang, D., Gui, J., & Song, L. T. 基于相干距离形状上下文的手部形状识别[J]. 模式识别, 2012, 45(9): 3348-3359.
[6] Kumar, A. 融入队列信息以实现可靠的掌纹认证[C]//2008年第六届印度计算机视觉、图形与图像处理会议（ICVGIP'08）. IEEE, 2008: 583-590.
[7] Ferrer, M. A., Morales, A., Travieso, C. M., & Alonso, J. B. 基于手部几何、掌纹与指纹纹理的低成本多模态生物特征识别系统[C]//2007年第41届IEEE国际卡汉安全技术会议. IEEE, 2007: 52-58.

我们基于本数据集提出了一种双流卷积神经网络（Convolutional Neural Network, CNN）用于性别分类任务。随后，我们将该训练完成的双流CNN用作特征提取器，同时应用于性别分类与生物特征识别任务。后者采用两种不同的方法实现：第一种方法中，我们从训练完成的CNN中提取深度特征，构建特征向量以训练支持向量机（Support Vector Machine, SVM）分类器；第二种方法中，三个SVM分类器分别接收从训练完成的CNN不同层提取的深度特征，同时我们使用局部二值模式（Local Binary Pattern, LBP）特征训练一个SVM分类器，通过整合所有SVM分类器的分类得分以提升正确识别率。您可从下表下载训练完成的模型与分类器。

由于我们的数据集存在女性手部图像数量偏多的问题（详见上文统计信息），我们为每个性别选取1000张背侧手部图像用于训练，500张背侧手部图像用于测试。图像选取过程为随机采样，且训练集与测试集的受试者群体互不重叠，即若某受试者的手部图像出现在训练集中，则该受试者不会出现在测试集中，反之亦然。掌侧手部图像的划分遵循相同规则。针对每一种拍摄视角，我们均重复实验10次以避免过拟合问题，并以准确率的平均值作为评估指标。

为进行对比实验，我们采用了10组训练-测试划分对，分别训练了多种图像分类方法，包括：(1) 视觉词袋（Bag of Visual Words, BoW）；(2) Fisher向量（Fisher Vector, FV）；(3) AlexNet（CNN）；(4) VGG-16（CNN）；(5) VGG-19（CNN）；(6) GoogLeNet（CNN）。针对前两种图像分类框架（BoW与FV），我们分别采用了三种不同的特征描述子：(1) SIFT（尺度不变特征变换，Scale-Invariant Feature Transform）；(2) C-SIFT；(3) rgSIFT。为便于后续对比，我们建议采用相同的评估准则。若需下载本实验中使用的10组训练-测试划分对，请参阅下表：

每组划分对包含以下文件：您可使用该MATLAB代码提取每次实验中使用的图像。该代码将生成10个目录，每个目录包含对应性别的训练集与测试集。随后您可使用imageDatastore函数加载这些数据集（详见CNN_training.m源代码）。

训练完成的CNN模型、SVM分类器与实验结果
若未安装MATLAB神经网络工具箱模型支持包，该函数将在Add-On Explorer中提供所需支持包的下载链接。如需安装支持包，请点击链接后点击"安装"按钮。您可在命令行输入模型名称（例如alexnet、vgg16、vgg19与googlenet）以验证安装是否成功。
* 要求MATLAB 2016或更高版本。
** 要求MATLAB 2017b或更高版本。
+ 基于我们的CNN模型作为特征提取器训练得到的SVM分类器，详见论文所述。该SVM分类器采用拼接后的特征向量进行训练，该特征向量由第一个流的fc9层特征、第二个流的fc10层特征与融合全连接层的特征拼接而成。基于LBP/SVM的分类器采用拼接后的特征向量进行训练，该特征向量由LBP特征、第一个流的fc9层特征、第二个流的fc10层特征与融合全连接层的特征拼接而成。

针对生物特征识别任务，我们采用了不同的训练-测试划分。针对80、100与120名受试者，我们为每只手的两种拍摄视角（掌侧或背侧）分别选取10张手部图像用于训练，4张用于测试。我们每次实验均随机选取受试者与图像，并重复实验10次，以平均识别准确率作为评估指标。为便于后续对比，我们建议采用相同的评估准则。若需下载本实验中使用的10组训练-测试划分对，请参阅下表：

每组划分对包含以下文件：您可使用该MATLAB代码提取每次实验中使用的图像。该代码将生成10个目录，每个目录包含对应受试者群体的训练集与测试集。每个文件名均包含受试者ID，例如0000000_Hand_0000055.jpg表示该图像属于受试者编号0000000，文件名其余部分为原始图像名称。您可使用该MATLAB代码加载所有图像文件名并提取对应的受试者ID。

训练完成的SVM分类器与实验结果
* 基于我们的CNN模型作为特征提取器训练得到的SVM分类器，详见论文所述。每个.mat文件均包含一个Classifier对象，其中：

疑问与评论可发送至：mafifi[at]eecs[dot]yorku[dot]ca 或 m.afifi[at]aun[dot]edu[dot]eg

搜集汇总

背景与挑战

背景概述

11kHands人手拍照数据集是一个包含11076张高分辨率手部图像的大型数据集，涵盖190名不同年龄、性别和肤色的受试者。每张图像附带详细的元数据，适用于性别识别和生物统计识别研究，具有学术免费使用的特点。