evaluation

Human-to-human communication via the computer is mainly done using a keyboard or microphone. In the field of Virtual Reality (VR), where the most immersive experience possible is desired, the use of a keyboard contradicts this goal, while the use of a microphone is not always desirable (e.g. silent commands during task force training) or simply not possible (e.g. if the user has a hearing loss). Data gloves help to increase immersion within the VR as they correspond to our natural interaction. At the same time, they offer the possibility to accurately capture hand shapes, such as those used in non-verbal communication (e.g. thumbs up, okay gesture, ...) and in sign language. In this paper, we present a hand shape recognition system using Manus Prime X data gloves, including data acquisition, data preprocessing, and data classification to enable nonverbal communication within VR. We investigate the impact on accuracy and classification time of using an Outlier Detection and a Feature Selection approach in our data preprocessing. To obtain a more generalized approach, we also studied the impact of artificial Data Augmentation, i.e., we create new artificial data from the recorded and filtered data to augment the training dataset. With our approach, 56 different hand shapes could be distinguished with an accuracy of up to 93.28%. With a reduced number of 27 hand shapes, an accuracy of up to 95.55% could be achieved. Voting Meta-Classifier (VL2) has proven to be the most accurate, albeit slowest, classifier. A good alternative is Random Forest (RF), which was even able to achieve better accuracy values in a few cases and was generally somewhat faster. Outlier Detection has proven to be a effective approach, especially in improving classification time. Overall, we have shown that our hand shape recognition system using data gloves is suitable for communication within VR.
64 different hand shapes were recorded from 20 participants, each with 3 repetitions. The files are structured as follows:

1st line Name of the hand shape
2nd line Left or right hand
3rd line empty
4th-23rd line Features Repetition 1
24th line empty
25th - 44th line Features repetition 2
45th line blank
46th - 65th line Features repetition 3

The features are ordered as follows:

Thumb spread
Index Finger spread
Middle Finger spread
Ring Finger spread
Pinky spread
Thumb stretch CMC
Thumb stretch MCP
Thumb stretch IP
Index Finger stretch MCP
Index Finger stretch PIP
Index Finger stretch DIP
Middle Finger stretch MCP
Middle Finger stretch PIP
Middle Finger stretch DIP
Ring Finger stretch MCP
Ring Finger stretch PIP
Ring Finger stretch DIP
Pinky stretch MCP
Pinky stretch PIP
Pinky stretch DIP

人与人之间通过计算机进行的通信主要依靠键盘或麦克风实现。在追求极致沉浸体验的虚拟现实（Virtual Reality，VR）领域中，键盘的使用与该沉浸目标相悖，而麦克风的应用则并非始终适宜——例如特遣部队训练中需下达静默指令的场景，或是用户存在听力障碍时无法使用麦克风的情况。
数据手套能够有效提升VR场景中的沉浸感，因其契合人类自然交互的模式。同时，数据手套可精准捕捉手部姿态，涵盖非语言交流中使用的各类手势（如点赞、OK手势等）以及手语手势。
本文提出了一套基于Manus Prime X数据手套的手部姿态识别系统，包含数据采集、数据预处理与数据分类三个模块，以实现VR场景内的非语言人际通信。我们探究了在数据预处理阶段引入异常值检测与特征选择方法对识别准确率及分类耗时的影响。为获得更具泛化性的解决方案，我们还研究了人工数据增强的作用：即通过已记录并预处理的原始数据生成新的人工样本，以扩充训练数据集。
经实验验证，本系统可区分56种手部姿态，最高识别准确率达93.28%；若将姿态类别缩减至27种，最高识别准确率可提升至95.55%。投票元分类器（Voting Meta-Classifier, VL2）被证实为准确率最高的分类器，尽管其分类速度最慢；随机森林（Random Forest, RF）则是性能优异的替代方案，在部分场景下甚至能取得更高的识别准确率，且整体分类速度略快。异常值检测方法被证实为有效的优化手段，尤其在缩短分类耗时方面效果显著。综上，我们证明了基于数据手套的手部姿态识别系统适用于VR场景下的人际通信。
本研究从20名参与者处采集了64种手部姿态的数据，每种姿态重复采集3次。数据文件的结构如下：
第1行：手部姿态名称
第2行：左手或右手
第3行：空行
第4行至第23行：第1次重复采集的特征数据
第24行：空行
第25行至第44行：第2次重复采集的特征数据
第45行：空行
第46行至第65行：第3次重复采集的特征数据

特征的排列顺序如下：
拇指外展
食指外展
中指外展
无名指外展
小指外展
拇指掌腕关节伸展
拇指掌指关节伸展
拇指指间关节伸展
食指掌指关节伸展
食指近节指间关节伸展
食指远节指间关节伸展
中指掌指关节伸展
中指近节指间关节伸展
中指远节指间关节伸展
无名指掌指关节伸展
无名指近节指间关节伸展
无名指远节指间关节伸展
小指掌指关节伸展
小指近节指间关节伸展
小指远节指间关节伸展