Embodied Neuromorphic Synergy for Lighting-robust Machine Vision [data]

In this research, inspired by the bio-principles governing the human pupillary control pathway, we devise and implement a novel neuromorphic exposure control (NEC) system. This innovation effectively alleviates the longstanding saturation problem that has plagued real-world intelligent machine vision systems operating under highly dynamic lighting conditions.The NEC resolves this challenge at its core by exploiting bio-principles found in peripheral vision to the computation of a novel trilinear event double integral (TEDI). This approach enables accurate connections between events and frames in the physics space for swift irradiance prediction, ultimately facilitating rapid control parameter updates.Our experimental results demonstrate the remarkable efficiency, low latency, superior generalization capability, and bio-inspired nature of the NEC in delivering timely and robust neuromorphic synergy for lighting-robust machine vision across a wide range of real-world applications. These applications encompass autonomous driving, mixed-reality, and three-dimensional reconstruction.

在本项研究中，受人类瞳孔控制通路所遵循的生物学原理启发，我们设计并实施了一种新颖的神经形态曝光控制（NEC）系统。该创新有效地缓解了长期困扰在高度动态光照条件下运行的实时智能机器视觉系统的饱和问题。NEC通过利用周围视觉中的生物学原理来计算一种新颖的三线性事件双重积分（TEDI），从而在核心上解决了这一挑战。这种方法实现了事件与物理空间中帧之间的准确连接，从而能够迅速预测辐照度，最终促进快速控制参数更新。我们的实验结果表明，NEC在提供及时且稳健的神经形态协同作用，以实现具有良好光照鲁棒性的机器视觉方面，具有卓越的效率、低延迟、优越的泛化能力以及生物启发特性。这些应用包括自动驾驶、混合现实和三维重建等众多实际应用领域。