Sashimi: A toolkit for facilitating high-throughput organismal image segmentation using deep learning

1. Digitized specimens are an indispensable resource for rapidly acquiring big datasets and typically must be preprocessed prior to conducting analyses. One crucial image preprocessing step in any image analysis workflow is image segmentation, or the ability to clearly contrast the foreground target from the background noise in an image. This procedure is typically done manually, creating a potential bottleneck for efforts to quantify biodiversity from image databases. Image segmentation meta-algorithms using deep learning provide an opportunity to relax this bottleneck. However, the most accessible pre-trained convolutional neural networks (CNNs) have been trained on a small fraction of biodiversity, thus limiting their utility. 2. We trained a deep learning model to automatically segment target fish from images with both standardized and complex, noisy backgrounds. We then assessed the performance of our deep learning model using qualitative visual inspection and quantitative image segmentation metrics of pixel overlap between reference segmentation masks generated manually by experts and those automatically predicted by our model. 3. Visual inspection revealed that our model segmented fishes with high precision and relatively few artifacts. These results suggest that the meta-algorithm (Mask R-CNN), in which our current fish segmentation model relies on, is well-suited for generating high-fidelity segmented specimen images across a variety of background contexts at rapid pace. 4. We present Sashimi, a user-friendly command line toolkit to facilitate rapid, automated high-throughput image segmentation of digitized organisms. Sashimi is accessible to non-programmers and does not require experience with deep learning to use. The flexibility of Mask R-CNN allows users to generate a segmentation model for use on diverse animal and plant images using transfer learning with training datasets as small as a few hundred images. To help grow the taxonomic scope of images that can be recognized, Sashimi also includes a central database for sharing and distributing custom-trained segmentation models of other unrepresented organisms. Lastly, Sashimi includes both auxiliary image preprocessing functions useful for some popular downstream color pattern analysis workflows, as well as a simple script to aid users in qualitatively and quantitatively assessing segmentation model performance for complementary sets of automatically and manually segmented images.

1. 数字化标本是快速获取大型数据集的核心资源，且在开展分析前通常需完成预处理。图像分析工作流中的关键预处理步骤之一为图像分割（image segmentation），即实现图像前景目标与背景噪声的清晰区分。该流程多依赖人工操作，成为从图像数据库中量化生物多样性工作的潜在瓶颈。基于深度学习的图像分割元算法为缓解这一瓶颈提供了可行方案。然而，当前主流的预训练卷积神经网络（Convolutional Neural Networks, CNNs）仅在极小部分生物类群的数据集上完成训练，因此限制了其应用潜力。
2. 本研究训练了一款深度学习模型，可自动从标准化背景与复杂带噪背景的图像中分割出目标鱼类。随后，我们通过定性目视检验与定量图像分割指标——即专家手动生成的参考分割掩码（segmentation masks）与模型自动预测掩码之间的像素重叠度——对该深度学习模型的性能进行了评估。
3. 目视检验结果显示，本模型对鱼类的分割精度较高，且产生的伪影较少。上述结果表明，本鱼类分割模型所依托的元算法（Mask R-CNN）能够快速生成高保真的分割后标本图像，且可适配多种背景环境。
4. 本研究推出了一款名为Sashimi的易用型命令行工具包，可实现数字化生物图像的快速自动化高通量图像分割。该工具包面向非编程人员，无需用户具备深度学习使用经验即可直接使用。依托Mask R-CNN的灵活性，用户仅需使用数百张图像规模的训练数据集，即可通过迁移学习（Transfer Learning）为多样化的动植物图像构建专属分割模型。为拓展可识别图像的分类学覆盖范围，Sashimi还内置了一个中央数据库，用于共享与分发其他未覆盖类群的自定义训练分割模型。此外，Sashimi集成了适用于部分主流下游色彩模式分析工作流的辅助图像预处理功能，同时提供了一款简易脚本，可辅助用户对自动与手动分割图像的互补数据集进行定性与定量的分割模型性能评估。