Energy-Efficient Visual Search by Eye Movements with a Low-Latency Spiking Neural Agent

Human vision incorporates a non-uniform resolution retina, efficient eye movement strategies, and spiking neural networks (SNNs) to balance requirements in visual field size, visual resolution, energy cost, and inference latency. However, whether these features can synergize to produce more energy-efficient computer vision algorithms with human-like eye movements remains underexplored. Here, we record human visual search data and establish an bio-inspired visual search model (BVSM). To our knowledge, BVSM is a rare SNN-based active visual search agent: it unifies non-uniform retinal sampling, event-driven spiking feature processing and memory, and a learned eye-movement policy within a single closed-loop framework, enabling low-latency saccade decisions in continuous space. The model combines a non-uniform resolution retina with spiking feature extraction, memory, and saccade decision modules, and employs population coding for fast saccade decisions. It can learn either a human-like or near-optimal fixation strategy by reinforcement learning, outperform human search performance, and achieve high energy efficiency through low saccade decision latency and sparse activation. Our findings suggest that SNNs with non-uniform resolution retina and efficient eye-movement strategies can facilitate more energy-efficient computer vision algorithms, especially when a large visual field, high visual resolution, and low energy cost are simultaneously required.

人类视觉系统具备非均匀分辨率视网膜（non-uniform resolution retina）、高效眼球运动策略以及脉冲神经网络（spiking neural networks, SNNs），以此在视野范围、视觉分辨率、能量消耗与推理延迟之间达成平衡。然而，这些特征能否协同作用，结合类人眼球运动开发更高效节能的计算机视觉算法，这一问题仍未得到充分探索。本研究通过记录人类视觉搜索数据，构建了一款受生物启发的视觉搜索模型（bio-inspired visual search model, BVSM）。据我们所知，BVSM是为数不多的基于SNN的主动视觉搜索智能体（active visual search agent）：它将非均匀视网膜采样、事件驱动脉冲特征处理与记忆，以及习得的眼球运动策略统一于单个闭环框架之中，可在连续空间中实现低延迟扫视决策。该模型将非均匀分辨率视网膜与脉冲特征提取、记忆及扫视决策模块相结合，并采用群体编码（population coding）实现快速扫视决策。该模型可通过强化学习（reinforcement learning）习得类人或近最优的注视策略，其搜索性能优于人类，且凭借低扫视决策延迟与稀疏激活特性实现了极高的能量利用效率。本研究结果表明，搭载非均匀分辨率视网膜与高效眼球运动策略的SNNs，可助力开发更节能的计算机视觉算法，尤其适用于同时需要大视野、高视觉分辨率与低能量消耗的场景。