Inductive biases of neural network modularity in spatial navigation

The brain may have evolved a modular architecture for reward-based learning in daily tasks, with circuits featuring functionally specialized modules that match the task structure. We propose that this architecture enables better learning and generalization than architectures with less specialized modules. To test this hypothesis, we trained reinforcement learning agents with various neural architectures on a naturalistic navigation task. We found that the architecture that largely segregates computations of state representation, value, and action into specialized modules enables more efficient learning and better generalization. The behavior of agents with this modular architecture also resembles macaque behaviors more closely. Investigating the latent state computations in these agents, we discovered that the learned state representation combines prediction and observation, weighted by their relative uncertainty, akin to a Kalman filter. These results shed light on the possible rationa..., , , # Inductive biases of neural network modularity in spatial navigation

This is a project exploring neural architectures for deep reinforcement learning agents.

All the codes have been tested on Windows machines with Anaconda and CUDA-capable GPUs. The following instructions allow users to run codes in this repo based on the Windows+CUDA GPU system that has been used. However, in general, MacOS or Linux machines, with or without GPUs, should also work with slight modifications in the setup.

## Setup

Follow these steps to set up the project:

### Download project repository

1. Download the project repository `Inductive_bias-v1.0.0.zip` from [Zenodo](https://zenodo.org/doi/10.5281/zenodo.10957521).
2. Unzip and navigate to the project folder; it should contain two subfolders: `analysis` and `model`.

### Download data

1. Download data `data.zip`.
2. Unzip the downloaded data and move the 'data' folder into the project's main folder.
3. Confirm that your project's folder now contains t...

大脑或许演化出了模块化架构，以支撑日常任务中的基于奖励的学习，其神经环路包含与任务结构相匹配的功能特化模块。我们提出，相较于特化程度更低的架构，此类架构能够实现更高效的学习与泛化性能。为验证这一假说，我们在一项自然主义导航任务中，对搭载多种神经架构的强化学习智能体（reinforcement learning agents）开展了训练。实验结果表明，将状态表征、价值与动作的计算大幅分离至不同特化模块的架构，能够实现更高效的学习与更优异的泛化能力。此类模块化架构智能体的行为也更接近猕猴的行为模式。通过探究这些智能体的潜在状态计算过程，我们发现，习得的状态表征结合了预测与观测信息，并以二者的相对不确定性作为权重进行融合，这与卡尔曼滤波器（Kalman filter）的运作原理相似。上述研究结果为空间导航领域神经网络模块化的归纳偏置（inductive biases）提供了潜在的理论阐释……
# 空间导航中神经网络模块化的归纳偏置

本项目旨在探究面向深度强化学习智能体的神经架构。

本项目所有代码均已在搭载Anaconda与支持CUDA的GPU的Windows设备上完成测试。下文将基于本次测试所用的Windows+CUDA GPU环境，为用户提供运行本仓库代码的操作指南。总体而言，无论是否配备GPU，MacOS或Linux系统仅需对安装流程稍作修改即可运行代码。

## 环境搭建

请按照以下步骤搭建本项目环境：

### 下载项目仓库
1. 从Zenodo平台[https://zenodo.org/doi/10.5281/zenodo.10957521]下载项目仓库压缩包`Inductive_bias-v1.0.0.zip`。
2. 解压压缩包并进入项目主文件夹，该文件夹应包含两个子文件夹：`analysis`与`model`。

### 下载数据集
1. 下载数据集压缩包`data.zip`。
2. 解压下载的数据集压缩包，并将解压得到的`data`文件夹移动至项目主文件夹内。
3. 确认项目主文件夹中现已包含……