Frame-Labeled 60 GHz FMCW Radar Gesture Dataset

As the field of human-computer interaction continues to evolve, there is a growing need for robust datasets that can enable the development of gesture recognition systems that operate reliably in diverse real-world scenarios. We present a radar-based gesture dataset, recorded using the BGT60TR13C XENSIV™ 60GHz Frequency Modulated Continuous Radar sensor to address this need. This dataset includes both nominal gestures and anomalous gestures, providing a diverse and challenging benchmark for understanding and improving gesture recognition systems. The dataset contains a total of 49,000 gesture recordings, with 25,000 nominal gestures and 24,000 anomalous gestures. Each recording consists of 100 frames of raw radar data, accompanied by a label file that provides annotations for every individual frame in each gesture sequence. This frame-based annotation allows for high-resolution temporal analysis and evaluation. Nominal Gesture Data The nominal gestures represent standard, correctly performed gestures. These gestures were collected to serve as the baseline for gesture recognition tasks. The details of the nominal data are as follows: Gesture Types: The dataset includes five nominal gesture types: Swipe Left Swipe Right Swipe Up Swipe Down Push Total Samples: 25,000 nominal gestures. Participants: The nominal gestures were performed by 12 participants (p1 through p12). Each nominal gesture has a corresponding label file that annotates every frame with the nominal gesture type, providing a detailed temporal profile for training and evaluation purposes. Anomalous Gesture Data The anomalous gestures represent deviations from the nominal gestures. These anomalies were designed to simulate real-world conditions in which gestures might be performed incorrectly, under varying speeds, or with modified execution patterns. The anomalous data introduces additional challenges for gesture recognition models, testing their ability to generalize and handle edge cases effectively. Total Samples: 24,000 anomalous gestures. Anomaly Types: The anomalous gestures include three distinct types of anomalies: Fast Executions: Gestures performed at a rapid pace, lasting approximately 0.1 seconds (much faster than the nominal average of 0.5 seconds). Slow Executions: Gestures performed at a significantly slower pace, lasting approximately 3 seconds (much slower than the nominal average). Wrist Executions: Gestures performed using the wrist instead of a fully extended arm, significantly altering the execution pattern. Participants: The anomalous gestures involved contributions from eight participants, including p1, p2, p6, p7, p9, p10, p11, and p12. Locations: All anomalous gestures were collected in location e1 (a closed-space meeting room). Radar Configuration Details The radar system was configured with an operational frequency range spanning from 58.5 GHz to 62.5 GHz. This configuration provides a range resolution of 37.5 mm and the ability to resolve targets at a maximum range of 1.2 meters. For signal transmission, the radar employed a burst configuration comprising 32 chirps per burst with a frame rate of 33 Hz and a pulse repetition time of 300 µs. Data Format The data for each user, categorized by location and anomaly type, is saved in compressed .npz files. Each .npz file contains key-value pairs for the data and its corresponding labels. The file naming convention is as follows:UserLabel_EnvironmentLabel(_AnomalyLabel).npy. For nominal gestures, the anomaly label is omitted. The .npz file contains two primary keys: inputs: Represents the raw radar data. targets: Refers to the corresponding label vector for the raw data. The raw radar data inputsis stored as a NumPy array with 5 dimensions, structured as follows:n_recordings x n_frames x n_antennas x n_chirps x n_samples, where: n_recordings: The number of gesture sequence instances (i.e., recordings). n_frames: The frame length of each gesture (100 frames per gesture). n_antennas: The number of virtual antennas (3 antennas). n_chirps: The number of chirps per frame (32 chirps). n_samples: The number of samples per chirp (64 samples). The labels targetsare stored as a NumPy array with 2 dimensions, structured as follows:n_recordings x n_frames, where: n_recordings: The number of gesture sequence instances (i.e., recordings). n_frames: The frame length of each gesture (100 frames per gesture). Each entry in the targets matrix corresponds to the frame-level label for the associated raw radar data in inputs. The total size of the dataset is approximately 48.1 GB, provided as a compressed file named radar_dataset.zip. Metadata The user labels are defined as follows: p1: Male p2: Female p3: Female p4: Male p5: Male p6: Male p7: Male p8: Male p9: Male p10: Female p11: Male p12: Male The environmental labels included in the dataset are defined as follows: e1: Closed-space meeting room e2: Open-space office room e3: Library e4: Kitchen e5: Exercise room e6: Bedroom The anomaly labels included in the dataset are defined as follows: fast: Fast gesture execution slow: Slow gesture execution wrist: Wrist gesture execution This dataset represents a robust and diverse set of radar-based gesture data, enabling researchers and developers to explore novel models and evaluate their robustness in a variety of scenarios. The inclusion of frame-based labeling provides an additional level of detail to facilitate the design of advanced gesture recognition systems that can operate with high temporal resolution. Disclaimer This dataset builds upon the version previously published on IEEE DataExplorer (https://ieee-dataport.org/documents/60-ghz-fmcw-radar-gesture-dataset), which included only one label per recording. In contrast, this version includes frame-based labels, providing individual annotations for each frame of the recorded gestures. By offering more granular labeling, this dataset further supports the development and evaluation of gesture recognition models with enhanced temporal precision. However, the raw radar data remains unchanged compared to the dataset available on IEEE DataExplorer.