Continuous Hand Movement Dataset for Daily Living Activities using MediaPipe

This dataset presents a continuous hand joint angle time-series dataset collected using a monocular RGB webcam and the MediaPipe Hands framework, designed to support research in human motion analysis, grasp recognition, biomechanics, and time-series forecasting. Motivation Understanding human hand movement is critical for a wide range of applications including human-computer interaction (HCI), robotics, rehabilitation systems, and assistive technologies. While several datasets exist for gesture recognition, most are limited to discrete gestures or image-based classification tasks. In contrast, this dataset focuses on continuous temporal dynamics of hand motion, enabling the development of models for sequence prediction, temporal segmentation, and motion forecasting. Data Collection Setup The dataset was collected using: A monocular RGB webcam The MediaPipe Hands framework for real-time hand landmark detection From the detected 3D landmarks, 11 anatomically meaningful joint angles were computed per frame using geometric relationships between landmark points. The dataset is recorded in a real-world environment, without specialized motion capture systems, making it highly relevant for practical AI applications. Dataset Structure The dataset includes recordings from: 10 right-handed subjects Each subject performs multiple sessions, where: Each session is recorded as a single continuous time-series Sessions include multiple trials and grip executions Temporal annotations are provided via: Trial_ID (trial segmentation) Grip_ID (grasp type segmentation) Joint Angle Features (11) The following joint angles are computed: Thumb_CMC Thumb_MCP Thumb_IP Index_MCP Index_PIP Middle_MCP Middle_PIP Ring_MCP Ring_PIP Little_MCP Little_PIP All angles are measured in degrees. Grip Types and Activities The dataset includes five common daily living grasp patterns: Power Grasp — Bottle Tripod Grip — Pen Static Power Hold — Phone The phone is lifted and held using a power grasp without thumb interaction Precision Pinch — Paper / Coin Lateral Pinch — Book Each grip is performed in a structured sequence within a session. Recording Protocol Each session follows a predefined sequence: Phase Action Duration Grip 1 Bottle (Power grasp) 5 s Transition Relax 2 s Grip 2 Pen (Tripod) 5 s Transition Relax 2 s Grip 3 Phone (Static hold) 5 s Transition Relax 2 s Grip 4 Paper/Coin (Pinch) 5 s Transition Relax 2 s Grip 5 Book (Lateral pinch) 5 s This structure enables: Clear segmentation Temporal modeling Transition analysis between grips Sampling and Format Sampling rate: ~30 FPS Units: Degrees Data format: Excel (.xlsx) Visualization: Session-wise trajectory plots (.png) Each Excel file contains: A Data sheet with time-series joint angles and annotations A Metadata sheet with session-level information Processing Pipeline Capture video frames from webcam Detect hand landmarks using MediaPipe Hands Compute joint angles using vector geometry Record continuous time-series data Annotate trials and grip transitions Store structured data in Excel format Generate visualization plots for analysis Key Contributions Continuous session-level dataset (rare in existing literature) Temporal annotations (Trial_ID, Grip_ID) Real-time capture setup (no specialized hardware) Joint angle representation (compact and interpretable) Suitable for both AI research and biomechanical analysis Potential Applications This dataset can be used for: Time-series forecasting (LSTM, GRU, Transformers) Hand gesture and grasp classification Human-computer interaction (HCI) systems Biomechanical analysis of hand motion Rehabilitation and assistive technologies Motion prediction and sequence modeling Limitations Limited to right-handed subjects Captured using a single RGB camera (no depth sensor) Possible noise due to landmark estimation errors Controlled environment (not fully unconstrained real-world settings) Future Work Inclusion of left-handed subjects Multi-view or depth-based recordings Higher sampling rates Expanded activity set Integration with force or EMG data Author and Acknowledgment Author: Shubham YadavComputer Science & Engineering, IIT Patna Jyotindra NarayanDepartment of Mechanical Engineering, IIT Patna Citation If you use this dataset, please cite: @dataset{yadav_narayan_hand_dataset_2026, author = {Shubham Yadav and Jyotindra Narayan}, title = {Continuous Hand Movement Dataset for Daily Living Activities using MediaPipe}, year = {2026}, school = {Department of Mechanical Engineering,Indian Institute of Technology Patna} }