BOLD5000 Additional ROIs and RDMs for neural network research

Artificial neural networks (ANNs) are sensitive to perturbations and
adversarial attacks. One hypothesized solution to adversarial robustness
is to align manifolds in the embedded space of neural networks with
biologically grounded manifolds. Recent state-of-the-art works that
emphasize learning robust neural representations, rather than optimizing
for a specific target task like classification, support the idea that
researchers should investigate this hypothesis.  While works have
shown that fine-tuning ANNs to coincide with biological vision does
increase robustness to both perturbations and adversarial attacks, these
works have relied on proprietary datasets- the lack of publicly available
biological benchmarks make it difficult to evaluate the efficacy of these
claims. Here, we deliver a curated dataset consisting of biological
representations of images taken from two commonly used computer vision
datasets, ImageNet and COCO, that can be easily integrated into model
training and evaluation. Specifically, we take a large functional magnetic
resonance imaging (fMRI) dataset (BOLD5000), preprocess it into
representational dissimilarity matrices (RDMs), and establish an
infrastructure that anyone can use to train models with biologically
grounded representations. Using this infrastructure, we investigate the
representations of several popular neural networks and find that as
networks have been optimized for tasks, their correspondence with
biological fidelity has decreased. Additionally, we use a previously
unexplored graph-based technique, Fiedler partitioning, to showcase the
viability of the biological data, and the potential to extend these
analyses by extending RDMs into Laplacian matrices. Overall, our findings
demonstrate the potential of utilizing our new biological benchmark to
effectively enhance the robustness of models.