participant_1

Human-to-human communication via the computer is mainly done using a keyboard or microphone. In the field of Virtual Reality (VR), where the most immersive experience possible is desired, the use of a keyboard contradicts this goal, while the use of a microphone is not always desirable (e.g. silent commands during task force training) or simply not possible (e.g. if the user has a hearing loss). Data gloves help to increase immersion within the VR as they correspond to our natural interaction. At the same time, they offer the possibility to accurately capture hand shapes, such as those used in non-verbal communication (e.g. thumbs up, okay gesture, ...) and in sign language. In this paper, we present a hand shape recognition system using Manus Prime X data gloves, including data acquisition, data preprocessing, and data classification to enable nonverbal communication within VR. We investigate the impact on accuracy and classification time of using an Outlier Detection and a Feature Selection approach in our data preprocessing. To obtain a more generalized approach, we also studied the impact of artificial Data Augmentation, i.e., we create new artificial data from the recorded and filtered data to augment the training dataset. With our approach, 56 different hand shapes could be distinguished with an accuracy of up to 93.28%. With a reduced number of 27 hand shapes, an accuracy of up to 95.55% could be achieved. Voting Meta-Classifier (VL2) has proven to be the most accurate, albeit slowest, classifier. A good alternative is Random Forest (RF), which was even able to achieve better accuracy values in a few cases and was generally somewhat faster. Outlier Detection has proven to be a effective approach, especially in improving classification time. Overall, we have shown that our hand shape recognition system using data gloves is suitable for communication within VR.
64 different hand shapes were recorded from 20 participants, each with 3 repetitions. The files are structured as follows:

1st line Name of the hand shape
2nd line Left or right hand
3rd line empty
4th-23rd line Features Repetition 1
24th line empty
25th - 44th line Features repetition 2
45th line blank
46th - 65th line Features repetition 3

The features are ordered as follows:

Thumb spread
Index Finger spread
Middle Finger spread
Ring Finger spread
Pinky spread
Thumb stretch CMC
Thumb stretch MCP
Thumb stretch IP
Index Finger stretch MCP
Index Finger stretch PIP
Index Finger stretch DIP
Middle Finger stretch MCP
Middle Finger stretch PIP
Middle Finger stretch DIP
Ring Finger stretch MCP
Ring Finger stretch PIP
Ring Finger stretch DIP
Pinky stretch MCP
Pinky stretch PIP
Pinky stretch DIP

人与人之间通过计算机进行的通信主要依赖键盘或麦克风。在追求极致沉浸体验的虚拟现实（Virtual Reality, VR）领域中，键盘的使用与该目标相悖，而麦克风的使用则并非始终适用——例如在特遣部队训练中需要静默指令时，或是用户存在听力障碍时完全无法使用。数据手套（data gloves）能够契合人类自然交互的模式，从而提升VR场景中的沉浸感；同时还可精准捕捉手部姿态，用于非语言沟通（如点赞、OK手势等）以及手语识别。
本文提出了一套基于Manus Prime X数据手套的手部姿态识别系统，涵盖数据采集、数据预处理与数据分类三个模块，以实现VR场景下的非语言通信。我们探究了在数据预处理阶段引入异常检测（Outlier Detection）与特征选择（Feature Selection）方法对识别精度与分类耗时的影响。为了获得泛化性更强的方案，我们还研究了人工数据增强（Data Augmentation）的作用——即通过已记录并预处理的数据生成新的人工样本，以扩充训练数据集。
通过所提方案，可区分56种不同手部姿态，最高识别精度可达93.28%；若将手部姿态类别缩减至27种，最高识别精度可提升至95.55%。其中，投票元分类器（Voting Meta-Classifier, VL2）被证实为精度最高的分类器，尽管其分类速度最慢；随机森林（Random Forest, RF）则是不错的替代方案，在部分场景下甚至能取得更高的识别精度，且整体耗时略短。异常检测（Outlier Detection）方法被证实十分有效，尤其在缩短分类耗时方面表现突出。综上，我们验证了基于数据手套的手部姿态识别系统适用于VR场景中的通信任务。
本次研究从20名参与者处采集了64种不同手部姿态的样本，每名参与者针对每种姿态完成3次重复采集。数据文件的结构如下：
第1行：手部姿态名称
第2行：左手或右手
第3行：空行
第4~23行：第1次重复的特征数据
第24行：空行
第25~44行：第2次重复的特征数据
第45行：空行
第46~65行：第3次重复的特征数据

特征的排序如下：
拇指展开度
食指展开度
中指展开度
无名指展开度
小指展开度
拇指CMC关节伸展度
拇指MCP关节伸展度
拇指IP关节伸展度
食指MCP关节伸展度
食指PIP关节伸展度
食指DIP关节伸展度
中指MCP关节伸展度
中指PIP关节伸展度
中指DIP关节伸展度
无名指MCP关节伸展度
无名指PIP关节伸展度
无名指DIP关节伸展度
小指MCP关节伸展度
小指PIP关节伸展度
小指DIP关节伸展度