mRI: multi-modal 3d human pose estimation dataset using mmwave, rgb-d, and inertial sensors|人体姿态估计数据集|多模态学习数据集

The ability to estimate 3D human body pose and movement, also known as human pose estimation~(HPE), enables many applications for home-based health monitoring, such as remote rehabilitation training. Several possible solutions have emerged using sensors ranging from RGB cameras, depth sensors, millimeter-Wave (mmWave) radars, and wearable inertial sensors. Despite previous efforts on datasets and benchmarks for HPE, few datasets exploit multiple modalities and focus on home-based health monitoring.
To bridge this gap, we present mRI, a multi-modal 3D human pose estimation dataset with mmWave, RGB-D, and Inertial Sensors. Our dataset consists of over 5 million frames from 20 subjects performing rehabilitation exercises and supports the benchmarks of HPE and action detection. We perform extensive experiments using our dataset and delineate the strength of each modality.
We hope that the release of mRI can catalyze the research in pose estimation, multi-modal learning, and action understan..., , , # mRI: multi-modal 3d human pose estimation dataset using mmwave, rgb-d, and inertial sensors

[Access this dataset on Dryad](https://doi.org/10.5061/dryad.9ghx3ffpp)

Data storage for NeurIPS 2022 paper [mRI: multi-modal 3d human pose estimation dataset using mmwave, rgb-d, and inertial sensors](https://proceedings.neurips.cc/paper_files/paper/2022/hash/af9c9c6d2da701da5a0acf91ec217815-Abstract-Datasets_and_Benchmarks.html)

Update 11/27/2023: We fixed the bug in the videos zip file.

## Data and file structure

There are two main sections of the data.

### RGB video data: blurred\_videos.zip

There are 40 videos for 20 subjects (left and right each).

### Other raw data, features, and model: dataset\_release.zip

The folder structure should be like this:

```
${ROOT}
|-- raw_data
| |-- imu
| |-- eaf_file
| |-- radar
| |-- unixtime
| |-- videolabels
|-- aligned_data
| |-- imu
| |-- radar
| |-- pose_labels
|-- features
| |-- imu
| |-- radar
|-- model
| |-- imu
| ...