userdependency

Human-to-human communication via the computer is mainly done using a keyboard or microphone. In the field of Virtual Reality (VR), where the most immersive experience possible is desired, the use of a keyboard contradicts this goal, while the use of a microphone is not always desirable (e.g. silent commands during task force training) or simply not possible (e.g. if the user has a hearing loss). Data gloves help to increase immersion within the VR as they correspond to our natural interaction. At the same time, they offer the possibility to accurately capture hand shapes, such as those used in non-verbal communication (e.g. thumbs up, okay gesture, ...) and in sign language. In this paper, we present a hand shape recognition system using Manus Prime X data gloves, including data acquisition, data preprocessing, and data classification to enable nonverbal communication within VR. We investigate the impact on accuracy and classification time of using an Outlier Detection and a Feature Selection approach in our data preprocessing. To obtain a more generalized approach, we also studied the impact of artificial Data Augmentation, i.e., we create new artificial data from the recorded and filtered data to augment the training dataset. With our approach, 56 different hand shapes could be distinguished with an accuracy of up to 93.28%. With a reduced number of 27 hand shapes, an accuracy of up to 95.55% could be achieved. Voting Meta-Classifier (VL2) has proven to be the most accurate, albeit slowest, classifier. A good alternative is Random Forest (RF), which was even able to achieve better accuracy values in a few cases and was generally somewhat faster. Outlier Detection has proven to be a effective approach, especially in improving classification time. Overall, we have shown that our hand shape recognition system using data gloves is suitable for communication within VR. 64 different hand shapes were recorded from 20 participants, each with 3 repetitions. The files are structured as follows: 1st line Name of the hand shape 2nd line Left or right hand 3rd line empty 4th-23rd line Features Repetition 1 24th line empty 25th - 44th line Features repetition 2 45th line blank 46th - 65th line Features repetition 3 The features are ordered as follows: Thumb spread Index Finger spread Middle Finger spread Ring Finger spread Pinky spread Thumb stretch CMC Thumb stretch MCP Thumb stretch IP Index Finger stretch MCP Index Finger stretch PIP Index Finger stretch DIP Middle Finger stretch MCP Middle Finger stretch PIP Middle Finger stretch DIP Ring Finger stretch MCP Ring Finger stretch PIP Ring Finger stretch DIP Pinky stretch MCP Pinky stretch PIP Pinky stretch DIP

通过计算机进行的人际交流主要依赖键盘或麦克风。在追求极致沉浸感的虚拟现实（Virtual Reality, VR）领域中，键盘的使用与该目标相悖，而麦克风的应用也并非始终适用——例如特遣部队训练中需要静音指令的场景，或是用户存在听力障碍时无法使用麦克风。数据手套（data gloves）能够提升VR场景的沉浸感，因其契合人类的自然交互方式；同时还可精准捕捉手部姿态，涵盖非语言交流（如点赞、OK手势等）与手语中的各类手势。
本文提出了一套基于Manus Prime X数据手套的手部姿态识别系统，涵盖数据采集、数据预处理与数据分类模块，以实现VR场景下的非语言交流。我们研究了异常检测（Outlier Detection）与特征选择（Feature Selection）方法在数据预处理阶段对识别精度及分类耗时的影响。为获得更具泛化性的解决方案，我们还探究了人工数据增强（Artificial Data Augmentation）的作用：即通过已记录并滤波后的原始数据生成新的人工样本，以此扩充训练数据集。
采用本方案可区分56种手部姿态，最高识别精度可达93.28%；若将姿态类别缩减至27种，最高识别精度可提升至95.55%。投票元分类器（Voting Meta-Classifier, VL2）被证明是精度最高但耗时最长的分类器；随机森林（Random Forest, RF）则是优秀的替代方案，在部分场景下甚至能取得更高的识别精度，且整体耗时更短。异常检测方法效果显著，尤其在缩短分类耗时方面表现突出。综上，我们证实了基于数据手套的手部姿态识别系统可适用于VR场景中的人际交流。
本次研究从20名受试者处采集了64种手部姿态的样本，每名受试者完成3次重复采集。数据文件的结构如下：
第1行：手部姿态名称
第2行：左手或右手
第3行：空行
第4-23行：第1次重复的特征数据
第24行：空行
第25-44行：第2次重复的特征数据
第45行：空行
第46-65行：第3次重复的特征数据
特征顺序如下：
1. 拇指外展（Thumb spread）
2. 食指外展（Index Finger spread）
3. 中指外展（Middle Finger spread）
4. 无名指外展（Ring Finger spread）
5. 小指外展（Pinky spread）
6. 拇指腕掌关节伸展（Thumb stretch CMC）
7. 拇指掌指关节伸展（Thumb stretch MCP）
8. 拇指指间关节伸展（Thumb stretch IP）
9. 食指掌指关节伸展（Index Finger stretch MCP）
10. 食指近节指间关节伸展（Index Finger stretch PIP）
11. 食指远节指间关节伸展（Index Finger stretch DIP）
12. 中指掌指关节伸展（Middle Finger stretch MCP）
13. 中指近节指间关节伸展（Middle Finger stretch PIP）
14. 中指远节指间关节伸展（Middle Finger stretch DIP）
15. 无名指掌指关节伸展（Ring Finger stretch MCP）
16. 无名指近节指间关节伸展（Ring Finger stretch PIP）
17. 无名指远节指间关节伸展（Ring Finger stretch DIP）
18. 小指掌指关节伸展（Pinky stretch MCP）
19. 小指近节指间关节伸展（Pinky stretch PIP）
20. 小指远节指间关节伸展（Pinky stretch DIP）