Finger gesture recognition with smart skin technology and deep learning

Finger gesture recognition was extensively studied in recent years for a wide range of human-machine interface applications. Surface electromyography (sEMG), in particular, is an attractive, enabling technique in the realm of finger gesture recognition, and both low and high-density sEMG were previously studied. Despite the clear potential, cumbersome electrode wiring and electronic instrumentation render contemporary sEMG-based finger gestures recognition to be performed under unnatural conditions. Recent developments in smart skin technology provide an opportunity to collect sEMG data in more natural conditions. Here we report on a novel approach based on a soft 16-electrode array, a miniature and wireless data acquisition unit and neural network analysis, in order to achieve gesture recognition under natural conditions. Finger gesture recognition accuracy values, as high as 93.1%, were achieved for 8 gestures when the training and test data were from the same session. For the first time, high accuracy values are also reported for training and test data from different sessions for three different hand positions. These results demonstrate an important step towards sEMG-based gesture recognition in non-laboratory settings, such as in gaming or Metaverse.

Methods
Eight healthy subjects (aged 18–30) completed two recording sessions with good SNR. Electrode arrays were placed on the region of the extensor digitorum muscle of the dominant hand. Muscle location was identified by applying strong abduction of the fingers. During the recording, each subject sat or stood in front of a table (depending on the position of the hand being examined). An instructional video displayed on a computer was used to guide the subjects.