Fine-Grained Sports, Yoga, and Dance Postures Recognition: A Benchmark Analysis

Abstract—Human body-pose estimation is a complex problem in computer vision. Recent research interests have been widened specifically on the Sports, Yoga, and Dance (SYD) postures for maintaining health conditions. The SYD pose categories are regarded as a fine-grained image classification task due to the complex movement of body parts. Deep Convolutional Neural Networks (CNNs) have attained significantly improved performance in solving various human body-pose estimation problems. Though decent progress has been achieved in yoga postures recognition using deep learning techniques, fine-grained sports, and dance recognition necessitates ample research attention. However, no benchmark public image dataset with sufficient inter-class and intra-class variations is available yet to address sports and dance postures classification. To solve this limitation, we have proposed two image datasets, one for 102 sport categories and another for 12 dance styles. Two public datasets, Yoga-82 which contains 82 classes and Yoga-107 represents 107 classes are collected for yoga postures. These four SYD datasets are experimented with the proposed deep model, SYD-Net, which integrates a patch-based attention (PbA) mechanism on top of standard backbone CNNs. The PbA module leverages the self-attention mechanism that learns contextual information from a set of uniform and multi-scale patches and emphasizes discriminative features to understand the semantic correlation among patches. Moreover, random erasing data augmentation is applied to improve performance. The proposed SYD-Net has achieved state-of-the-art accuracy on Yoga-82 using five base CNNs. SYD-Net’s accuracy on other datasets is remarkable, implying its efficiency. Our Sports-102 and Dance-12 datasets are publicly available for research only.

Please cite this work:

A. Bera, M. Nasipuri, O. Krejcar and D. Bhattacharjee, "Fine-Grained Sports, Yoga, and Dance Postures Recognition: A Benchmark Analysis," in IEEE Transactions on Instrumentation and Measurement, vol. 72, pp. 1-13, 2023, Art no. 5020613, doi: 10.1109/TIM.2023.3293564.

https://ieeexplore.ieee.org/document/10177209

摘要——人体姿态估计是计算机视觉领域的一项复杂问题。近年来，研究兴趣已扩展至具体针对健康维护的体育、瑜伽和舞蹈（SYD）姿态。由于身体部位运动的复杂性，SYD姿态类别被视为一种细粒度的图像分类任务。深度卷积神经网络（CNNs）在解决各种人体姿态估计问题方面取得了显著性能提升。尽管在利用深度学习技术进行瑜伽姿态识别方面已取得一定的进展，但细粒度的体育和舞蹈识别仍需充分的研究关注。然而，目前尚无包含足够类间和类内变异性基准的公共图像数据集来处理体育和舞蹈姿态分类。为解决这一限制，我们提出了两个图像数据集，一个包含102个体育类别，另一个包含12种舞蹈风格。收集了包含82个类别的Yoga-82和包含107个类别的Yoga-107两个公共数据集，用于瑜伽姿态。这四个SYD数据集与所提出的深度模型SYD-Net进行了实验，该模型在标准骨干CNNs之上集成了基于补丁的注意力（PbA）机制。PbA模块利用自注意力机制，从一组统一的多尺度补丁中学习上下文信息，并强调区分性特征以理解补丁之间的语义相关性。此外，应用随机擦除数据增强以提升性能。所提出的SYD-Net在Yoga-82数据集上使用五种基础CNNs实现了最先进的准确率。SYD-Net在其他数据集上的准确率也相当显著，这表明其效率。我们的Sports-102和Dance-12数据集仅限研究目的公开可用。