Intracortical brain-computer interface for navigation in virtual reality in macaque monkeys

We present an innovative intracortical Brain-Computer Interface (BCI) to bridge the gap between laboratory settings and real-world applications. This BCI approach introduces three key advancements. First, we utilized neural signals from three macaque brain regions – primary motor, dorsal and ventral premotor cortex – enabling precise and flexible decoding of real-time three-dimensional (3D) sphere/avatar velocities. Second, we developed a realistic, immersive 3D virtual reality setup with dynamic camera tracking, allowing continuous navigation and obstacle avoidance that closely mimic real-world scenarios. Finally, our BCI approach is very well suited for use by paralyzed patients, featuring a brief passive fixation without overt movements and closed-loop operation without retraining of the decoder during online decoding, relying on the user’s neural plasticity and the decoder’s robust generalization across tasks. Our BCI adapted to different environments, targets, and obstacles, illustrating its potential to substantially enhance the quality of life for paralyzed patients by enabling natural, reliable and flexible control in complex settings.

我们提出了一种创新性的皮层内脑机接口（intracortical Brain-Computer Interface, BCI），用以弥合实验室场景与现实世界应用之间的差距。该BCI方案实现了三项核心技术突破：其一，我们采集了猕猴三个脑区的神经信号——初级运动皮层、背侧前运动皮层与腹侧前运动皮层，可实现对实时三维（3D）球体/虚拟化身运动速度的精准且灵活解码；其二，我们搭建了搭载动态相机追踪的高拟真沉浸式三维虚拟现实系统，能够实现高度贴近现实场景的连续导航与避障操作；其三，本BCI方案极适配瘫痪患者使用，仅需用户进行短暂的被动注视而无需做出显性动作，且在线解码过程中无需对解码器重新训练，可借助用户的神经可塑性与解码器跨任务的强泛化能力稳定运行。本BCI可适配不同环境、目标与障碍物，表明其能够通过让瘫痪患者在复杂场景中实现自然、可靠且灵活的控制，大幅提升患者的生活质量，展现出巨大的应用潜力。