In-sensor multilevel image adjustment for high-clarity contour extraction using adjustable synaptic phototransistors

Robotic vision has traditionally relied on high-performance yet
resource-intensive computing solutions, which necessitate
high-throughput data transmission from vision sensors to
remote computing servers, sacrificing energy-efficiency and
processing speed. A promising solution is data compaction through
contour extraction, visualizing only the outlines of objects while
eliminating superfluous backgrounds. Here, we introduce an
in-sensor multi-level image adjustment method using adjustable
synaptic phototransistors, enabling the capture of well-defined images
with optimal brightness and contrast suitable for achieving
high-clarity contour extraction. This is enabled by emulating
dopamine-mediated neuronal excitability regulation
mechanisms. Electrostatic gating effect either facilitates or
inhibits time-dependent photocurrent accumulation, adjusting
photo-responses to varying lighting conditions. Through excitatory and
inhibitory modes, the adjustable synaptic phototransistor
enhances visibility of dim and bright regions,
respectively, facilitating distinct contour extraction and
high-accuracy semantic segmentation. Evaluations using road
images demonstrate improvement of both object detection accuracy and
Intersection over Union, and significant compression of data
volume.