participant_19

Human-to-human communication via the computer is mainly done using a keyboard or microphone. In the field of Virtual Reality (VR), where the most immersive experience possible is desired, the use of a keyboard contradicts this goal, while the use of a microphone is not always desirable (e.g. silent commands during task force training) or simply not possible (e.g. if the user has a hearing loss). Data gloves help to increase immersion within the VR as they correspond to our natural interaction. At the same time, they offer the possibility to accurately capture hand shapes, such as those used in non-verbal communication (e.g. thumbs up, okay gesture, ...) and in sign language. In this paper, we present a hand shape recognition system using Manus Prime X data gloves, including data acquisition, data preprocessing, and data classification to enable nonverbal communication within VR. We investigate the impact on accuracy and classification time of using an Outlier Detection and a Feature Selection approach in our data preprocessing. To obtain a more generalized approach, we also studied the impact of artificial Data Augmentation, i.e., we create new artificial data from the recorded and filtered data to augment the training dataset. With our approach, 56 different hand shapes could be distinguished with an accuracy of up to 93.28%. With a reduced number of 27 hand shapes, an accuracy of up to 95.55% could be achieved. Voting Meta-Classifier (VL2) has proven to be the most accurate, albeit slowest, classifier. A good alternative is Random Forest (RF), which was even able to achieve better accuracy values in a few cases and was generally somewhat faster. Outlier Detection has proven to be a effective approach, especially in improving classification time. Overall, we have shown that our hand shape recognition system using data gloves is suitable for communication within VR. 64 different hand shapes were recorded from 20 participants, each with 3 repetitions. The files are structured as follows: 1st line Name of the hand shape 2nd line Left or right hand 3rd line empty 4th-23rd line Features Repetition 1 24th line empty 25th - 44th line Features repetition 2 45th line blank 46th - 65th line Features repetition 3 The features are ordered as follows: Thumb spread Index Finger spread Middle Finger spread Ring Finger spread Pinky spread Thumb stretch CMC Thumb stretch MCP Thumb stretch IP Index Finger stretch MCP Index Finger stretch PIP Index Finger stretch DIP Middle Finger stretch MCP Middle Finger stretch PIP Middle Finger stretch DIP Ring Finger stretch MCP Ring Finger stretch PIP Ring Finger stretch DIP Pinky stretch MCP Pinky stretch PIP Pinky stretch DIP

通过计算机实现的人际交互，主流输入方式为键盘与麦克风。在追求极致沉浸感的虚拟现实（Virtual Reality, VR）场景中，键盘输入与该目标相悖，而麦克风输入也并非始终适用——例如特遣部队训练中需静默下达指令时，或是用户存在听力障碍无法/不便使用麦克风的场景下。数据手套（Data Gloves）贴合人类自然交互习惯，可有效提升VR场景沉浸感，同时能够精准采集手部姿态，适配非语言交流（如点赞、OK手势等）以及手语场景的手势识别需求。
本文提出一种基于Manus Prime X数据手套的手部姿态识别系统，涵盖数据采集、数据预处理与数据分类模块，以实现VR场景下的非语言交互。本研究探究了在数据预处理阶段引入异常值检测与特征选择方法，对识别准确率与分类耗时的影响。为构建更具泛化性的解决方案，我们还研究了人工数据增强（Artificial Data Augmentation）的影响：即通过已录制并滤波后的原始数据生成新的人工样本，以扩充训练数据集。
基于所提方案，本系统可区分56种手部姿态，最高识别准确率达93.28%；当缩减至27种姿态类别时，最高准确率可提升至95.55%。经实验验证，投票元分类器（Voting Meta-Classifier, VL2）是准确率最高的分类器，但耗时最长；而随机森林（Random Forest, RF）则是性能优异的替代方案，在部分场景中甚至可实现更高的识别准确率，且整体耗时略短。异常值检测方法被证实为高效的优化手段，尤其可显著缩短分类耗时。综上，本研究证实基于数据手套的手部姿态识别系统，可适配VR场景下的人际交互需求。
本数据集共采集20名受试者的64种手部姿态数据，每种姿态均重复录制3次。数据集文件结构如下：
第1行：手部姿态名称
第2行：左手/右手标识
第3行：空行
第4~23行：第1次重复的特征数据
第24行：空行
第25~44行：第2次重复的特征数据
第45行：空行
第46~65行：第3次重复的特征数据
特征数据的排列顺序如下：
拇指外展幅度、食指外展幅度、中指外展幅度、无名指外展幅度、小指外展幅度
拇指腕掌关节（Carpometacarpal, CMC）伸展度、拇指掌指关节（Metacarpophalangeal, MCP）伸展度、拇指指间关节（Interphalangeal, IP）伸展度
食指掌指关节（MCP）伸展度、食指近侧指间关节（Proximal Interphalangeal, PIP）伸展度、食指远侧指间关节（Distal Interphalangeal, DIP）伸展度
中指掌指关节（MCP）伸展度、中指近侧指间关节（PIP）伸展度、中指远侧指间关节（DIP）伸展度
无名指掌指关节（MCP）伸展度、无名指近侧指间关节（PIP）伸展度、无名指远侧指间关节（DIP）伸展度
小指掌指关节（MCP）伸展度、小指近侧指间关节（PIP）伸展度、小指远侧指间关节（DIP）伸展度