Inductive biases of neural network modularity in spatial navigation

The brain may have evolved a modular architecture for reward-based learning in daily tasks, with circuits featuring functionally specialized modules that match the task structure. We propose that this architecture enables better learning and generalization than architectures with less specialized modules. To test this hypothesis, we trained reinforcement learning agents with various neural architectures on a naturalistic navigation task. We found that the architecture that largely segregates computations of state representation, value, and action into specialized modules enables more efficient learning and better generalization. The behavior of agents with this modular architecture also resembles macaque behaviors more closely. Investigating the latent state computations in these agents, we discovered that the learned state representation combines prediction and observation, weighted by their relative uncertainty, akin to a Kalman filter. These results shed light on the possible rationa..., , , # Inductive biases of neural network modularity in spatial navigation

This is a project exploring neural architectures for deep reinforcement learning agents.

All the codes have been tested on Windows machines with Anaconda and CUDA-capable GPUs. The following instructions allow users to run codes in this repo based on the Windows+CUDA GPU system that has been used. However, in general, MacOS or Linux machines, with or without GPUs, should also work with slight modifications in the setup.

## Setup

Follow these steps to set up the project:

### Download project repository

1. Download the project repository `Inductive_bias-v1.0.0.zip` from [Zenodo](https://zenodo.org/doi/10.5281/zenodo.10957521).
2. Unzip and navigate to the project folder; it should contain two subfolders: `analysis` and `model`.

### Download data

1. Download data `data.zip`.
2. Unzip the downloaded data and move the 'data' folder into the project's main folder.
3. Confirm that your project's folder now contains t...