Abductees-Rescue: 3D Hand Landmarks Dataset for Real-Time Abduction Detection in Surveillance Systems

This work has been comprised of creating a 3D hand landmarks dataset for real-time detection of abduction cases through surveillance systems using the MediaPipe Hands framework. The dataset has been represented in two categories of 3D hand landmark gestures: normal hand gestures labeled with a "0" classlable and specific gestures indicating potential abduction cases labeled with a "1" classlable. The 3D hand landmarks have been extracted after applying MediaPipe on 4545 videos of normal hand gestures and 4566 videos of abduction cases hand gestures. These videos of hand gestures have been cropped from custom-recorded videos by volunteers specifically for creating this dataset using a developed algorithm for this purpose. Each cropped video contains 45 frames. Each frame contains one hand represented in 21 keypoint landmarks providing X, Y, and Z coordinates, creating a three-dimensional representation of the hand pose (palm and fingers) in this frame. These custom-videos have been recorded using surveillance cameras mounted at a standard 3-meter height across various environmental conditions, including different lighting scenarios (day/night) and distances up to 17 meters from the surveillance camera. The developping algorithm has been applied to track and crop hand gestures from the custom-videos and save them into new videos as mentioned appove and categorized into two groups 0 and 1. Augmentation techniques have been applied on the new videos to enhance dataset diversity via geometric transformations and visual quality adjustments.Then, the MediaPipe hands framwork had applying on the cropped videos to exctact the 3D hand landmarks. Finally, applying normalization tehcnique on the resulting PKL file of hands landmarks that contain all hands landmarks with their classlables (0 and 1), normalized to range [-1 : 1] to let the researchers deal directly with this dataset in bulding their models of machine learning etc,. Interested researchers could utilize this dataset to develop and train AI models for identifying potential abduction cases through hand gesture recognition in surveillance systems. The data is particularly valuable in scenarios where verbal communication is not feasible. Moreover, the structure of this dataset allows for direct application in training deep learning algorithms or feature extraction for machine learning models, enabling the development of real-time surveillance systems capable of recognizing sign requests for help through hand gestures.本研究构建了一套三维手部关键点数据集，旨在借助MediaPipe手部关键点框架（MediaPipe Hands），通过监控系统实时识别疑似绑架类场景。该数据集涵盖两类三维手部关键点手势：标记为类别标签"0"的正常手部手势，以及标记为类别标签"1"的疑似绑架类特定手势。 研究人员通过针对本数据集开发的自定义算法，从志愿者录制的原始视频中裁剪出手部手势视频：其中正常手部手势视频共4545段，疑似绑架类手势视频共4566段。每段裁剪后的视频包含45帧，每帧内包含单只手的21个关键点，涵盖X、Y、Z三维坐标，由此构建该帧内手部姿态（手掌与手指）的三维表征。 本研究采集的原始自定义视频由安装在标准3米高度的监控摄像头录制，涵盖多种环境条件，包括不同光照场景（日间/夜间），以及监控摄像头与拍摄对象之间最远17米的拍摄距离。本研究开发的算法可从原始视频中追踪并裁剪出手部手势，生成前述分类为0和1的新视频片段。随后通过几何变换与视觉质量调整等数据增强技术，提升数据集的多样性。接着将MediaPipe手部关键点框架应用于裁剪后的视频，提取三维手部关键点。最后对包含所有手部关键点及其类别标签（0与1）的PKL格式结果文件进行归一化处理，将坐标范围归一化至[-1, 1]区间，方便研究人员直接使用该数据集开展机器学习模型构建等相关工作。 感兴趣的研究人员可利用该数据集开发并训练AI模型，通过监控系统中的手部手势识别来识别疑似绑架类求助场景。该数据集在口头沟通不可行的场景中具备特殊应用价值。此外，该数据集的结构可直接用于深度学习算法训练或机器学习模型的特征提取，助力开发能够通过手部手势识别求助信号的实时监控系统。