DataSheet1_A novel approach for automatic annotation of human actions in 3D point clouds for flexible collaborative tasks with industrial robots.docx

Manual annotation for human action recognition with content semantics using 3D Point Cloud (3D-PC) in industrial environments consumes a lot of time and resources. This work aims to recognize, analyze, and model human actions to develop a framework for automatically extracting content semantics. Main Contributions of this work: 1. design a multi-layer structure of various DNN classifiers to detect and extract humans and dynamic objects using 3D-PC preciously, 2. empirical experiments with over 10 subjects for collecting datasets of human actions and activities in one industrial setting, 3. development of an intuitive GUI to verify human actions and its interaction activities with the environment, 4. design and implement a methodology for automatic sequence matching of human actions in 3D-PC. All these procedures are merged in the proposed framework and evaluated in one industrial Use-Case with flexible patch sizes. Comparing the new approach with standard methods has shown that the annotation process can be accelerated by 5.2 times through automation.

在工业环境中，基于3D点云（3D Point Cloud，3D-PC）开展带内容语义的人体动作识别相关手动标注工作，往往耗费大量时间与资源。本研究旨在对人体动作进行识别、分析与建模，以构建一套可自动提取内容语义的研究框架。本研究的主要贡献如下：1. 设计了由多种深度神经网络（Deep Neural Network, DNN）分类器组成的多层结构，可基于3D-PC精准检测并提取人体与动态目标；2. 开展了针对10余名受试者的实证实验，在单一工业场景下采集人体动作与活动数据集；3. 开发了一款直观的图形用户界面（Graphical User Interface, GUI），用于验证人体动作及其与环境的交互活动；4. 设计并实现了一种针对3D-PC中人体动作的自动序列匹配方法。上述所有流程均被整合至所提出的框架中，并在采用灵活点云补丁尺寸的单一工业用例中完成了评估。将本研究提出的新方法与标准方法进行对比后发现，通过自动化可将标注流程提速5.2倍。