MCFGes

Dataset Description

The dataset consists of 105 classes and 2264 samples, collected using an RGB camera and two IWR6843 ISK mmWave radars. The data is stored in a structured format designed to facilitate multimodal and cross-domain few-shot gesture recognition research.

Data Organization

The dataset file pickle.pkl is organized into a dictionary where each key corresponds to a class_idx. The value associated with each key is a list containing samples from that class.

dataset/│├── class_001/│   ├── sample_0001│   ├── sample_0002│   ...│├── class_002/│   ├── sample_0001│   ├── sample_0002│   ...│...

Each sample is a dictionary with the following keys:

mm_cloud1: Numpy array data from Radar 1, as described in the associated paper.

mm_cloud1_mask: Mask data indicating real gesture data versus padding (zeros) for Radar 1.

label: The label identifying the gesture class.

mm_cloud2: Numpy array data from Radar 2.

mm_cloud2_mask: Mask data indicating real gesture data versus padding (zeros) for Radar 2.

vis_rgb: Video data from the RGB camera.

vis_rgb_mask: Mask data indicating real gesture data versus padding (zeros) for the RGB video.

Dataset loading

All data undergoes preprocessing and is loaded into batches suitable for model input via the fs_dataset.py script, ensuring that the data is ready for use in machine learning models.

Spilit Table

The six cross-domain split tables are:

split_table_micro.json for MicroGesture(MG)

spilt_table_meeting.json for MeetingRoom(MR)

split_table_out_door.json for Outdoor(OD)

split_table_home.json for Home(H)

split_table_static.json for VR(V)

split_table_multi_peope.json (MP)

Design Patterns in the Project

Singleton Pattern: Utilized to manage parameter modules consistently across the project. Readers can adjust settings or introduce new configurations in the config directory to initiate different experiments.

Builder Pattern: This pattern is used to enable easy expansion of components such as the training engine, dataset, and model. By adding new build methods, users can enhance the project's capabilities and adapt it to new requirements.

Factory Pattern: Supports the registration of new models through decorators in various factories, allowing for the broadening of experimental scope and flexibility.

Overall Benefits: These design patterns are implemented to streamline the construction and expansion of experiments, promoting efficient development and scalable architecture.

Run:

You can edit the script.py and add config file to run your own experiment.

Others:

There are some git configs and pycache i forget to delete before uploading. Just delete or ignore them.

Some __init__.py files contains some import will not be used, just delete them either.