An optoelectronic artificial spiking neuron array with biomimetic spike-temporal pattern for in-sensor visual prediction

Humans can efficiently predict transient events with a low latency of approximately 80 ms. Simulating this capability to achieve accurate and rapid transient prediction at the edge is essential for advancing bionic machine vision. This task is, however, challenging due to the intricate nonlinear characteristics of dynamic motion, which requires high-throughput data processing and training on digital computers. In this work, we report an optoelectronic artificial neuron (OAN) array to implement in-sensor prediction without training and high-throughput data processing, achieved by emulating spike-temporal patterns of human vision for nonlinear spike encoding and memory. The OAN can nonlinearly encode light intensity into first-spike time (ranging from 7.37 to 0.24 $\upmu$s) and has a memory ability of 60 s due to oxygen vacancy dynamics. The nonlinear perception and memory capabilities spontaneously generate a spike-recurrent spatiotemporal information equation in situ, allowing the OAN arrays to predict future states using limited data without training. As a result, the array can rapidly and accurately predict pedestrian motion with a high executable frame rate ($\sim$125 fps) and low root mean square error ($\sim$0.014). Combined with spiking neural networks, the array achieves 100% accuracy in pedestrian avoidance. Our research provides forward-looking solutions for advancing bionic machine vision.