NEST: Spatially-mapped cell-cell communication patterns using a deep learning-based attention mechanism

Dysregulation of communication between cells mediates complex diseases such as cancer and diabetes. However, detecting cell-cell communication (CCC) at scale remains one of the greatest challenges in transcriptomics. While gene expression remains one of the greatest challenges in transcriptomics. While gene expression measured with single-cell RNA sequencing and spatial transcriptomics reinvigorated computational approaches to detecting CCC, most existing methods exhibit high false positive rates, do not integrate spatial proximity of ligand-receptor interactions, and cannot detect CCC between individual cells. We overcome these challenges by presenting NEST (NEural network on Spatial Transcriptomics), which uses a graph attention network paired with an unsupervised contrastive learning approach to decipher patterns of communication while retaining the strength of each signal. We introduce new synthetic benchmarking experiments which demonstrate how NEST outperforms existing tools and detects biologically-relevant CCC along with directionality and confidence across spot- and cell-based technologies measuring several different tissues and diseases. In our applications, NEST identifies T-cell homing signals in human lymph nodes, aggressive cancer CCC in lung adenocarcinoma, and discovers new patterns of communication in pancreatic cancer. Beyond two-dimensional data, we also highlight NEST's ability to detect CCC in three-dimensional spatial transcriptomic data. Two patient samples with resectable (Stage IIB) PDAC were included (Male, 76 years old and Female, 83 years old). Both patient samples were collected from the University Heath Network Biospecimens Program (Toronto, Canada). Ethical approval was obtained through the University Health Network Research Ethics Board (13-6377).