Persistent Memory in Single Node Delay-Coupled Reservoir Computing

Delays are ubiquitous in biological systems, ranging from genetic regulatory networks and synaptic conductances, to predator/pray population interactions. The evidence is mounting, not only to the presence of delays as physical constraints in signal propagation speed, but also to their functional role in providing dynamical diversity to the systems that comprise them. The latter observation in biological systems inspired the recent development of a computational architecture that harnesses this dynamical diversity, by delay-coupling a single nonlinear element to itself. This architecture is a particular realization of Reservoir Computing, where stimuli are injected into the system in time rather than in space as is the case with classical recurrent neural network realizations. This architecture also exhibits an internal memory which fades in time, an important prerequisite to the functioning of any reservoir computing device. However, fading memory is also a limitation to any computation that requires persistent storage. In order to overcome this limitation, the current work introduces an extended version to the single node Delay-Coupled Reservoir, that is based on trained linear feedback. We show by numerical simulations that adding task-specific linear feedback to the single node Delay-Coupled Reservoir extends the class of solvable tasks to those that require nonfading memory. We demonstrate, through several case studies, the ability of the extended system to carry out complex nonlinear computations that depend on past information, whereas the computational power of the system with fading memory alone quickly deteriorates. Our findings provide the theoretical basis for future physical realizations of a biologically-inspired ultrafast computing device with extended functionality.

延迟现象在生物系统中无处不在，涵盖基因调控网络、突触传导，乃至捕食者-猎物（predator/prey）种群交互等诸多场景。越来越多的证据表明，延迟不仅是信号传播速度限制带来的物理约束，还能为所属系统提供动态多样性并发挥功能性作用。生物系统中的这一发现，启发了近期一种计算架构的开发：该架构通过将单个非线性元件（nonlinear element）进行延迟自耦合，以此利用这种动态多样性。该架构是储备池计算（Reservoir Computing）的一种特定实现：与经典循环神经网络按空间维度注入刺激的实现方式不同，它以时序方式向系统注入输入信号。该架构同时具备随时间衰减的内部记忆，这是所有储备池计算设备正常运行的重要前提。然而，记忆衰减特性也会对需要持久存储的计算任务形成限制。为克服这一局限，本研究提出了基于训练线性反馈的单节点延迟耦合储备池（Delay-Coupled Reservoir）扩展版本。我们通过数值模拟证明，为单节点延迟耦合储备池添加任务专属的线性反馈，可将可求解任务的范围拓展至需要非衰减记忆的场景。我们通过多个案例研究证明，相较于仅具备衰减记忆、计算能力会快速退化的系统，该扩展系统能够完成依赖过往信息的复杂非线性计算任务。本研究结果为未来开发具备扩展功能、受生物系统启发的超高速计算设备提供了理论基础。