MCFGes

Dataset Description The dataset consists of 105 classes and 2264 samples, collected using an RGB camera and two IWR6843 ISK mmWave radars. The data is stored in a structured format designed to facilitate multimodal and cross-domain few-shot gesture recognition research. Data Organization The dataset file pickle.pkl is organized into a dictionary where each key corresponds to a class_idx. The value associated with each key is a list containing samples from that class. dataset/ │ ├── class_001/ │ ├── sample_0001 │ ├── sample_0002 │ ... │ ├── class_002/ │ ├── sample_0001 │ ├── sample_0002 │ ... │ ... Each sample is a dictionary with the following keys: mm_cloud1: Numpy array data from Radar 1, as described in the associated paper. mm_cloud1_mask: Mask data indicating real gesture data versus padding (zeros) for Radar 1. label: The label identifying the gesture class. mm_cloud2: Numpy array data from Radar 2. mm_cloud2_mask: Mask data indicating real gesture data versus padding (zeros) for Radar 2. vis_rgb: Video data from the RGB camera. vis_rgb_mask: Mask data indicating real gesture data versus padding (zeros) for the RGB video. Dataset loading All data undergoes preprocessing and is loaded into batches suitable for model input via the fs_dataset.py script, ensuring that the data is ready for use in machine learning models. Spilit Table The six cross-domain split tables are: split_table_micro.json for MicroGesture(MG) spilt_table_meeting.json for MeetingRoom(MR) split_table_out_door.json for Outdoor(OD) split_table_home.json for Home(H) split_table_static.json for VR(V) split_table_multi_peope.json (MP) Design Patterns in the Project Singleton Pattern: Utilized to manage parameter modules consistently across the project. Readers can adjust settings or introduce new configurations in the config directory to initiate different experiments. Builder Pattern: This pattern is used to enable easy expansion of components such as the training engine, dataset, and model. By adding new build methods, users can enhance the project's capabilities and adapt it to new requirements. Factory Pattern: Supports the registration of new models through decorators in various factories, allowing for the broadening of experimental scope and flexibility. Overall Benefits: These design patterns are implemented to streamline the construction and expansion of experiments, promoting efficient development and scalable architecture. Run: You can edit the script.py and add config file to run your own experiment. Others: There are some git configs and pycache i forget to delete before uploading. Just delete or ignore them. Some __init__.py files contains some import will not be used, just delete them either.

数据集说明
该数据集包含105个类别与2264个样本，采用RGB相机（RGB）与两台IWR6843 ISK毫米波雷达（mmWave radars）采集得到。数据采用结构化格式存储，旨在为多模态跨域少样本（Few-shot）手势识别研究提供支撑。

数据组织形式
数据集文件pickle.pkl为字典结构，每个键对应一个类别索引（class_idx），键值为该类别下的样本列表。
数据集目录结构如下：
dataset/
│
├── class_001/
│ ├── sample_0001
│ ├── sample_0002
│ ...
│
├── class_002/
│ ├── sample_0001
│ ├── sample_0002
│ ...
│
...
每个样本为字典格式，包含以下字段：
mm_cloud1：来自雷达1的NumPy数组（NumPy）数据，详细说明参见相关论文。
mm_cloud1_mask：雷达1的掩码数据，用于区分有效手势数据与补零（padding）的无效数据。
label：标识手势类别的标签。
mm_cloud2：来自雷达2的NumPy数组（NumPy）数据。
mm_cloud2_mask：雷达2的掩码数据，用于区分有效手势数据与补零的无效数据。
vis_rgb：RGB相机采集的视频数据。
vis_rgb_mask：RGB视频的掩码数据，用于区分有效手势数据与补零的无效数据。

数据集加载
所有数据已完成预处理，可通过fs_dataset.py脚本加载为适配模型输入的批量数据，确保可直接用于机器学习模型的训练与推理任务。

划分表
共包含6个跨域划分表：
- "split_table_micro.json" 对应微手势（MicroGesture, MG）任务
- "split_table_meeting.json" 对应会议室场景（MeetingRoom, MR）任务
- "split_table_out_door.json" 对应户外场景（Outdoor, OD）任务
- "split_table_home.json" 对应居家场景（Home, H）任务
- "split_table_static.json" 对应VR场景（VR, V）任务
- "split_table_multi_peope.json" 对应多人场景（Multi People, MP）

项目设计模式
单例模式（Singleton Pattern）：用于统一管理项目中的参数模块，开发者可通过config目录调整配置或新增配置项以启动不同实验。
建造者模式（Builder Pattern）：用于实现组件（如训练引擎、数据集、模型）的便捷扩展，通过新增构建方法即可拓展项目功能以适配新需求。
工厂模式（Factory Pattern）：支持通过各工厂的装饰器注册新模型，拓宽实验范围并提升灵活性。
整体优势：上述设计模式的应用简化了实验的构建与扩展流程，助力高效开发与可扩展的架构设计。

运行方式
可通过编辑script.py并添加配置文件来运行自定义实验。

其他说明
上传前遗留了部分Git配置文件与__pycache__目录，可直接删除或忽略；部分__init__.py文件包含未使用的导入语句，同样可删除。

搜集汇总

背景与挑战

背景概述

MCFGes是一个多模态手势识别数据集，包含105个类别和2264个样本，数据来自RGB摄像头和两个毫米波雷达。数据集以结构化格式组织，支持跨域少样本学习研究。