3D object detection in industrial site point clouds

Detection of three dimensional (3D) objects in point clouds is a challenging problem. Existing methods either focus on a specific type of object or scene, or require prior segmentation, both of which are usually inapplicable on real-world industrial applications. ❧ This thesis describe three methods to tackle the problem, with gradually improving performance and efficiency. The first is a general purpose 3D object detection method that combines Adaboost with 3D local features, without requirement for prior object segmentation. Experiments demonstrated competitive accuracy and robustness to occlusion, but this method suffers from limited rotation invariance. As an improvement, another method is presented with a multi-view detection approach that projects the 3D point clouds into several 2D depth images from multiple viewpoints, transforming the 3D problem into a series of 2D problems, which reduces complexity, stabilizes performance, and achieves rotation invariance. The problem is the huge amount of projected views and rotations that need to be individually detected, limiting the complexity and performance of 2D algorithm choice. Thus the third method is proposed to solve this with the introduction of convolutional neural network, because it can handle all viewpoints and rotations for the same class of object together, as well as predicting multiple classes of objects with the same network, without the need for individual detector for each object class. The detection efficiency is further improved by concatenating two extra levels of early rejection networks with binary outputs before the multi-class detection network. ❧ 3D object detection in point clouds is crucial for 3D industrial point cloud modeling. Prior efforts focus on primitive geometry, street structures or indoor objects, but industrial data has rarely been pursued. We integrate several algorithm components into an automatic 3D modeling system for industrial site point clouds, including modules for pipe modeling, plane classification and object detection, and solves the technology gaps revealed during the integration. The integrated system is able to produce classified models of large and complex industrial scenes with a quality that outperforms leading commercial software and comparable to professional hand-made models. ❧ This thesis also describes an earlier work in multi-modal image matching which inspires later research in 3D object detection by 2D projections. Most existing 2D descriptors only work well on images of a single modality with similar texture. This proposal presents a novel basic descriptor unit called a Gixel, which uses an additive scoring method to sample surrounding edge information. Several Gixels in a circular array create the Gixel Array Descriptor, excelling in multi-modal image matching with dominant line features.点云（point clouds）中的三维（3D）目标检测是一项极具挑战性的任务。现有方法要么仅针对特定类型的目标或场景，要么需要预先进行分割，这两类方案在实际工业场景中通常均不适用。 本论文提出三种解决方案以应对该问题，其性能与效率逐步提升。第一种为通用型三维目标检测方法，将自适应增强算法（Adaboost）与三维局部特征相结合，无需预先对目标进行分割。实验结果表明，该方法在遮挡情况下具备优异的检测精度与鲁棒性，但存在旋转不变性不足的缺陷。作为改进方案，第二种方法采用多视角检测思路：将三维点云从多个视角投影为若干二维深度图像，将三维检测问题转化为一系列二维检测问题，从而降低了算法复杂度、稳定了检测性能并实现了旋转不变性。但该方法的问题在于，需要对大量投影视角与旋转角度逐一进行检测，这限制了二维算法的选型空间与最终性能。因此，本文提出第三种方法，引入卷积神经网络（convolutional neural network, CNN）解决上述问题：卷积神经网络可同时处理同一类别目标的所有视角与旋转情况，且可通过同一网络完成多类别目标的检测，无需为每个目标类别单独设计检测器。此外，在多类别检测网络之前串联两个额外的二元输出早期拒绝网络，进一步提升了检测效率。 点云中的三维目标检测对于三维工业点云建模至关重要。此前的研究多聚焦于基础几何形态、街道场景或室内目标，针对工业场景数据的研究则较为匮乏。本文将多种算法组件集成至面向工业现场点云的自动化三维建模系统中，包括管道建模、平面分类与目标检测模块，并解决了集成过程中暴露的技术短板。该集成系统可对大型复杂工业场景生成分类化模型，其建模质量优于主流商用软件，且可媲美专业人工制作的模型。 本论文还介绍了一项早期的多模态图像匹配研究工作，该工作为后续基于二维投影的三维目标检测研究提供了灵感。现有多数二维描述子（descriptor）仅在纹理相似的单模态图像上表现优异。本文提出一种名为Gixel的新型基础描述子单元，其采用累加评分方法对周边边缘信息进行采样。将若干Gixel以环形阵列排布，即可构成Gixel阵列描述子，该描述子在以线条特征为主的多模态图像匹配任务中表现出色。