HoMG dataset

With the rapid development of augmented reality(AR) and virtual reality (VR) technology, human-computer interaction (HCI) has been greatly improved for gaming interaction of AR and VR control. The finger micro-gesture is a hot research focus due to the growth of the Internet of Things (IoT) and wearable technologies and recently Google has developed a radar based micro-gesture sensor which is Google Soli. Also, there are a number of finger micro-gesture techniques have been developed using Time of Flight (ToF) imaging sensors for wearable 3D glasses such as Atheer mobile glasses. The principle of holoscopic 3D (H3D) imaging mimics fly’s eye technique that captures a true 3D optical model of the scene using a microlens array, however, there is a limited progress of holoscopic 3D systems due to the lack of high quality public available database. In this paper, holoscopic 3D camera is used to capture high quality holoscopic 3D micro-gesture video images and a new unique holoscopic 3D micro-gesture (HoMG) database is produced. HoMG database recorded the image sequence of 3 conventional gestures from 40 participants under different settings and conditions. For the purpose of H3D micro-gesture recognition, HoMG has a video subset of 960 videos and a still image subset with 30635 images. Initial micro-gesture recognition on both subsets has been conducted using the traditional 2D image and video features and popular classifiers and some encouraging performance has been achieved. The database will be available for the research communities and speed up the research in the area of holoscopic 3D micro-gesture.

随着增强现实（AR）与虚拟现实（VR）技术的快速发展，面向AR/VR操控的游戏交互场景中，人机交互（HCI）体验得到了显著提升。得益于物联网（IoT）与可穿戴技术的蓬勃发展，手指微手势成为当前的研究热点；近期谷歌还研发了基于雷达原理的微手势传感器谷歌Soli（Google Soli）。此外，针对Atheer移动眼镜等可穿戴3D眼镜，已有诸多基于飞行时间（ToF）成像传感器的手指微手势技术被提出。全视三维（H3D）成像原理模仿复眼技术，通过微透镜阵列捕获场景的真实三维光学模型，但由于缺乏高质量公开数据集，全视三维系统的研究进展较为有限。本文采用全视三维相机采集高质量的全视三维微手势视频图像，构建了全新的专属全视三维微手势（HoMG）数据集。HoMG数据集记录了40名参与者在多种设置与实验条件下的3种常规手势的图像序列。面向全视三维微手势识别任务，该数据集包含960个视频的视频子集，以及包含30635张静态图像的静态图像子集。研究团队已采用传统二维图像与视频特征结合主流分类器，对两个子集开展了初步的微手势识别实验，并取得了令人鼓舞的识别性能。本数据集将面向全球科研社区开放，以期推动全视三维微手势相关领域的研究进展。