Summary of spatial transcriptome datasets.

Spatial transcriptomics has gained popularity over the past decade due to its ability to evaluate transcriptome data while preserving spatial information. Cell segmentation is a crucial step in spatial transcriptomic analysis, as it enables the avoidance of unpredictable tissue disentanglement steps. Although high-quality cell segmentation algorithms can aid in the extraction of valuable data, traditional methods are frequently non-spatial, do not account for spatial information efficiently, and perform poorly when confronted with the problem of spatial transcriptome cell segmentation with varying shapes. In this study, we propose ST-CellSeg, an image-based machine learning method for spatial transcriptomics that uses manifold for cell segmentation and is novel in its consideration of multi-scale information. We first construct a fully connected graph which acts as a spatial transcriptomic manifold. Using multi-scale data, we then determine the low-dimensional spatial probability distribution representation for cell segmentation. Using the adjusted Rand index (ARI), normalized mutual information (NMI), and Silhouette coefficient (SC) as model performance measures, the proposed algorithm significantly outperforms baseline models in selected datasets and is efficient in computational complexity.

空间转录组学（Spatial transcriptomics）在过去十年间愈发流行，因其可在保留空间信息的同时对转录组数据进行评估。细胞分割是空间转录组分析的关键步骤，能够规避难以预测的组织解离步骤。尽管高质量的细胞分割算法有助于提取有价值的研究数据，但传统方法往往不具备空间感知能力，无法高效利用空间信息，且在应对形态多样的空间转录组细胞分割任务时表现不佳。本研究提出ST-CellSeg，一种面向空间转录组学的基于图像的机器学习方法，其采用流形（manifold）开展细胞分割，并创新性地融入了多尺度信息考量。我们首先构建全连接图以作为空间转录组流形；随后借助多尺度数据，确定用于细胞分割的低维空间概率分布表征。以调整兰德指数（adjusted Rand index, ARI）、归一化互信息（normalized mutual information, NMI）以及轮廓系数（Silhouette coefficient, SC）作为模型性能评估指标，所提算法在选定数据集上的表现显著优于基线模型，且具有较低的计算复杂度。