PkSLMNM: Pakistan Sign Language Manual and Non-Manual Gestures Dataset

Cite This Article S. Javaid and S. Rizvi, "A novel action transformer network for hybrid multimodal sign language recognition," Computers, Materials & Continua, vol. 74, no.1, pp. 523–537, 2023. https://doi.org/10.32604/cmc.2023.031924 Sign language is a non-verbal form of communication used by people with impaired hearing and speech. They also use facial actions to provide sign language prosody, similar to intonation in spoken languages. Sign Language Recognition (SLR) using hand signs is a typical way, however, face expression and body language play an important role in communication, which has not been analyzed to its fullest potential. In this paper, we present a dataset that comprises manual (hand signs) and non-manual (facial expressions and body movements) gestures of Pakistan Sign Language (PSL). It contains videos of 7 basic affective expressions performed by 100 healthy individuals, presented in an easily accessible format of .MP4 that can be used to train and test systems to make robust models for real-time applications using videos. Current data can also help with facial feature detection, classification of subjects by gender and age, or provide insights into any individual’s interest and emotional state.

引用本文：S. Javaid与S. Rizvi，《面向混合多模态手语识别的新型动作Transformer网络》，刊载于《Computers, Materials & Continua》2023年第74卷第1期，页码范围523–537，DOI：10.32604/cmc.2023.031924。手语是听障与言语障碍群体使用的非言语沟通载体，使用者还会通过面部动作传递手语韵律，其功能与口语中的语调相仿。基于手部动作的手语识别（Sign Language Recognition, SLR）是当前主流的识别方案，但面部表情与肢体语言在沟通中同样具备关键作用，这一点尚未得到充分的挖掘与分析。本文所提出的巴基斯坦手语（Pakistan Sign Language, PSL）数据集，涵盖手动（手部动作）与非手动（面部表情及肢体动作）两类手势。该数据集包含100名健康受试者完成的7种基础情感表达视频，采用通用易用的.MP4格式存储，可用于训练与测试视频识别系统，以构建适用于实时应用场景的鲁棒模型。此外，本数据集还可应用于面部特征检测、基于性别与年龄的受试者分类，亦可助力分析个体的兴趣倾向与情绪状态。