mmWave-based Activity Recognition Dataset

Description:

This mmWave Datasets are used for activity verification. It contains two datasets. The first dataset (FA Dataset) contains 14 common daily activities. This second one (EA Dataset) contains 5 kinds of eating activities. The data are captured by the mmWave radar TI-AWR1642. The dataset can be used by fellow researchers to reproduce the original work or to further explore other machine-learning problems in the domain of mmWave signals.

Format: .png format

Section 1: Device Configuration

A commodity mmWave radar TI AWR1642, which integrates a 2 × 4 antenna array. The detailed information of it can be found at https://www.ti.com/product/AWR1642#:~:text=The%20AWR1642%20is%20an%20ideal,of%2076%20to%2081%20GHz.

A TI DCA1000EVM data capture card is used to collect data from the mmWave device and send data to a laptop. The detailed information can be found at https://www.ti.com/tool/DCA1000EVM?keyMatch=DCA1000EVM.

mmWave radar work at the frequency in the range of 77~81GHz. The sampling rate is fixed at 100 frames per second and each frame has 17 chirps.

Section 2: Data Format

We provide our mmWave data in heatmaps for the two datasets. The data file is in the png format. The details are shown in the following:

FA Dataset

2 participants are included in the FA Dataset.

14 activities are included in the FA Dataset.

FA_d_p_i_u_j.png:

d represents the data to collect the data.

p represents the environment to collect the data.

i represents activity type index

u represents user id

j represents sample index

Example:

FA_20220101_lab_1_2_3 represents the 3rd data sample of user 2 of activity 1 collected in the lab

EA Dataset

2 participants are included in the EA Dataset.

5 activities are included in the EA Dataset.

EA_d_p_i_u_j.png:

d represents the data to collect the data.

p represents the environment to collect the data.

i represents the activity type index

u represents the user id

j represents the sample index

Section 3: Experimental Setup

FA Dataset

We place the mmWave device on a table with a height of 60cm.

The participants are asked to perform fitness activity in front of a mmWave device with a distance of 2m.

The data are collected at an lab with a size of (5.0m×3.0m).

EA Dataset

We place the mmWave device on a table with a height of 60cm.

The participants are asked to eat with different utensils (i.e., fork, fork&knife, spoon, chopsticks, bare hand) in front of a mmWave device with a distance of 1m.

The data are collected at an lab with a size of (5.0m×3.0m).

Section 4: Data Description

We develop a spatial-temporal heatmap to integrates multiple activity features, including the range of movement, velocity, and time duration of each activity repetition.

We first derive the Doppler-range map of the users’ activity by calculating Range-FFT and Doppler-FFT. Then, we generate the spatial-temporal heatmap by accumulating the velocity of every distance in every Doppler-range map together. Next, we normalize the derived velocity information and present the velocity-distance relationship in time dimension. In this way, we transfer the original instantaneous velocity-distance relationship to a more comprehensive spatial-temporal heatmap which describes the process of a whole activity.

As shown in Figure attached, in each spatial-temporal heatmap, the horizontal axis represents the time duration of an activity repetition while the vertical axis represents the range of movement. The velocity is represented by color.

We create 2 folders to store two dataset respectively. In FA folder, there are 14 subfolders, each contains repetitions from the same fitness activity. In EA folder, there are 5 subfolders, each contains repetitions with different utensils.

14 common daily activities and their corresponding folders

Folder Name

Activity Type

Folder Name

Activity Type

FA1

Crunches

FA8

Squats

FA2

Elbow plank and reach

FA9

Burpees

FA3

Leg raise

FA10

Chest squeezes

FA4

Lunges

FA11

High knees

FA5

Mountain climber

FA12

Side leg raise

FA6

Punches

FA13

Side to side chops

FA7

Push ups

FA14

Turning kicks

5 eating activities and their corresponding folders

Folder Name

Activity Type

EA1

Eating with chopsticks

EA2

Eating with fork

EA3

Eating with bare hand

EA4

Eating with fork&knife

EA5

Eating with spoon

Section 5: Raw Data and Data Processing Algorithms

We also provide the mmWave raw data (.mat format) stored in the same folder corresponding to the heatmap datasets. Each .mat file can store one set of activity repetitions (e.g., 4 repetations) from a same user.

For example: EA_d_p_i_u_j.mat:

d represents the data to collect the data.

p represents the environment to collect the data.

i represents the activity type index

u represents the user id

j represents the set index

We plan to provide the data processing algorithms (heatmap_generation.py) to load the mmWave raw data and generate the corresponding heatmap data.

Section 6: Citations

If your paper is related to our works, please cite our papers as follows.

https://ieeexplore.ieee.org/document/9868878/

Xie, Yucheng, Ruizhe Jiang, Xiaonan Guo, Yan Wang, Jerry Cheng, and Yingying Chen. "mmFit: Low-Effort Personalized Fitness Monitoring Using Millimeter Wave." In 2022 International Conference on Computer Communications and Networks (ICCCN), pp. 1-10. IEEE, 2022.

Bibtex:

@inproceedings{xie2022mmfit,

  title={mmFit: Low-Effort Personalized Fitness Monitoring Using Millimeter Wave},

  author={Xie, Yucheng and Jiang, Ruizhe and Guo, Xiaonan and Wang, Yan and Cheng, Jerry and Chen, Yingying},

  booktitle={2022 International Conference on Computer Communications and Networks (ICCCN)},

  pages={1--10},

  year={2022},

  organization={IEEE}

}

https://www.sciencedirect.com/science/article/abs/pii/S2352648321000532

Xie, Yucheng, Ruizhe Jiang, Xiaonan Guo, Yan Wang, Jerry Cheng, and Yingying Chen. "mmEat: Millimeter wave-enabled environment-invariant eating behavior monitoring." Smart Health 23 (2022): 100236.

Bibtex:

@article{xie2022mmeat,

  title={mmEat: Millimeter wave-enabled environment-invariant eating behavior monitoring},

  author={Xie, Yucheng and Jiang, Ruizhe and Guo, Xiaonan and Wang, Yan and Cheng, Jerry and Chen, Yingying},

  journal={Smart Health},

  volume={23},

  pages={100236},

  year={2022},

  publisher={Elsevier}

}