基于单帧图像的情感与社交意图手势识别数据

通过构建一个包含“比心”、“加油”、“勾引”等多种表达情感态度与社交邀请的静态手势图像，并为其标注对应意图类别的数据集，用于训练能够深度理解人类非语言社交信号的分类模型。该数据适用于高度拟人化的交互场景，例如直播互动中，观众通过手势触发“送爱心”、“加油打气”等虚拟礼物或实时弹幕特效；在元宇宙或VR社交平台中，用户的虚拟化身可以实时同步做出这些社交手势，与朋友进行更自然的互动；以及在AI虚拟伴侣或数字人应用中，用户可以通过手势与虚拟形象建立情感连接，获得更具人格化的回应。利用该数据训练的模型能够识别人类交流中更深层次的情感和社交意图，解决了传统手势识别大多局限于功能性指令（如控制、选择）的难题
面向情感与社交意图的静态手势识别旨在将单个图像中的手势分类为预定义的用于表达情感和社交的手势类别。具体过程包括：（1）数据收集：采集覆盖多种常见手势的图像，记录所属类别。（2）数据处理：利用手掌检测模型提取手部区域图片I_hand，然后将该区域输入到一个特征提取网络中,用来在高维特征空间表征手部区域。特征提取通过公式 F_gesture​=Encodercnn​(I_hand) 完成，其中F_gesture是代表手势语义的特征向量。（3）模型构建：在提取的特征向量后连接一个分类器Classifier，根据公式 P_class​=Classifier(F_gesture) 预测出类别，其中P_class是预测类别；关键评估指标包括平均分类准确率。

This dataset is constructed by collecting static hand gesture images that convey various emotional attitudes and social invitations, including finger heart gestures, cheering gestures, and flirtatious gestures, and annotating each image with its corresponding intent category, for training classification models that can deeply comprehend human non-verbal social signals. This dataset is applicable to highly anthropomorphic interaction scenarios. For instance, in live streaming interactions, viewers can trigger virtual gifts such as "sending love hearts", "cheering up" and real-time barrage effects via gestures; in metaverse or VR social platforms, users' virtual avatars can perform these social gestures in real-time synchronization to interact with friends more naturally; and in AI virtual companion or digital human applications, users can establish emotional connections with virtual avatars through gestures and acquire more personalized responses. Models trained on this dataset can recognize deeper emotional and social intentions in human communication, addressing the limitation of traditional gesture recognition that mostly focuses on functional commands such as control and selection.

Static gesture recognition for emotional and social intentions aims to classify gestures in a single image into pre-defined gesture categories that convey emotions and social intentions. The specific process includes:
(1) Data Collection: Collect images covering a variety of common gestures and record their respective categories.
(2) Data Processing: Use a palm detection model to extract the hand region image $I_{ ext{hand}}$, then input this region into a feature extraction network to represent the hand region in a high-dimensional feature space. Feature extraction is completed via the formula $F_{ ext{gesture}} = Encoder_{ ext{cnn}}(I_{ ext{hand}})$, where $F_{ ext{gesture}}$ is the feature vector representing gesture semantics.
(3) Model Construction: Connect a classifier to the extracted feature vector, and predict the category via the formula $P_{ ext{class}} = Classifier(F_{ ext{gesture}})$, where $P_{ ext{class}}$ is the predicted category; the key evaluation metrics include average classification accuracy.

搜集汇总

背景与挑战

背景概述

该数据集专注于情感与社交意图的手势识别，包含多种静态手势图像（如“比心”、“加油”），并标注了对应类别和语义特征向量，用于训练深度理解人类非语言社交信号的分类模型。它适用于高度拟人化的交互场景，如直播互动、元宇宙社交和AI虚拟伴侣，解决了传统手势识别局限于功能性指令的难题，平均分类准确率达到0.88。