Millimeter-wave Multi-View Radar (MMVR) Dataset

Introduction

The Millimeter-wave Multi-View Radar (MMVR) dataset comprises 345K multi-view radar frames collected from 25 human subjects over 6 different rooms. It includes 446K annotated bounding boxes/segmentation instances, and 7.59 million annotated keypoints to support three perception tasks: object detection, pose estimation, and instance segmentation in the image plane.

MMVR consists of 35 data sessions, each stored in a data folder with the format dxsy, where x represents the day and y the session index of that day when the data were collected.

Within each data folder, data frames are grouped into 395 non-overlapping one-minute data segments. Each data segment is stored in a folder named using a three-digit, zero-filled convention based on the chronological order. For example, the first one-minute data segment is saved in the folder 000, while the second one-minute data segment is stored in the folder 001.

Within each data segment folder, each data frame consists of 5 NPZ files: meta, radar, bounding boxes (bbox), keypoints (pose) and segmentation masks (mask). Each data frame is named using a five-digit, zero-filled convention based on the chronological order within the one-minute data segment.

Root/
├── d1s1/
│ └── ...
├── d1s2/
│ ├── 000/
│ │ ├── 00000_meta.npz ... Meta info
│ │ ├── 00000_radar.npz ... Horizontal/Vertical heatmaps
│ │ ├── 00000_bbox.npz ... 2D Bounding Boxes
│ │ ├── 00000_pose.npz ... 2D keypoints
│ │ ├── 00000_mask.npz ... 2D Segmentation masks
│ │ ├── 00001_meta.npz
│ │ ├── 00001_radar.npz
│ │ ├── 00001_bbox.npz
│ │ ├── 00001_pose.npz
│ │ ├── 00001_mask.npz
│ │ .
│ │ ├── 00899_meta.npz
│ │ ├── 00899_radar.npz
│ │ ├── 00899_bbox.npz
│ │ ├── 00899_pose.npz
│ │ └── 00899_mask.npz
│ └── 001/
│ ├── 00000_meta.npz
│ ├── 00000_radar.npz
│ ├── 00000_bbox.npz
│ ├── 00000_pose.npz
│ ├── 00000_mask.npz
. └── ...
├── d9s5/
│ └── ...
└── d9s6/
└── ...

At a Glance

The size of the unzipped dataset is ~81.8 GB

The MMVR dataset consists of 345K data frames with each frame including 5 NPZ files: meta, radar, bounding boxes (bbox), keypoints (pose) and segmentation masks (mask).

The MMVR dataset is divided into four smaller chunks to facilitate easier downloads: three chunks of approximately 20 GB each and one chunk of 12 GB.

P1.zip (24.2 GB):

README.md and its related figure folder (figs)

load_sample.ipynb: a python snippet to load and visualize a data frame

data_split.npz: an npz file containing the predefined data splits (S1 or S2) for training, validation, and test sets of data segments under P1 and P2.

all data frames under P1 (d1s1, d1s2, d2s2, d3s1, d3s2, d4s1)

P2_00.zip (20.4 GB): all data frames in d5s1 – d5s6 and d6s1 – d6s6

P2_01.zip (21.6 GB): all data frames in d7s1 – d7s5 and d8s1 – d8s6

P2_02.zip (11.7 GB): all data frames in d9s1 – d9s6

Citation

If you use the MMVR dataset in your research, please cite our contribution:

@inproceedings{MMVR2024,
title={MMVR: Millimeter-wave Multi-View Radar Dataset and Benchmark for Indoor Perception},
author={M. Mahbubur Rahman and Ryoma Yataka and Sorachi Kato and Pu Perry Wang and Peizhao Li and Adriano Cardace and Petros Boufounos},
booktitle={Proceedings of European Conference on Computer Vision (ECCV)},
pages={},
year={2024}
}

License

The MMVR dataset is released under CC-BY-SA-4.0 license.

All data:

Created by Mitsubishi Electric Research Laboratories (MERL), 2023

SPDX-License-Identifier: CC-BY-SA-4.0