A Multispectral Automated Transfer Technique (MATT) for machine-driven image labeling utilizing the Segment Anything Model (SAM)

Segment Anything Model (SAM) is drastically accelerating the speed and accuracy ofautomatically segmenting and labeling large Red-Green-Blue (RGB) imagery datasets. However, SAM isunable to segment and label images outside of the visible light spectrum, for example, for multispectral orhyperspectral imagery. Therefore, this paper outlines a method we call the Multispectral Automated TransferTechnique (MATT). By transposing SAM segmentation masks from RGB images we can automaticallysegment and label multispectral imagery with high precision and efficiency. For example, the resultsdemonstrate that segmenting and labeling a 2,400-image dataset utilizing MATT achieves a time reductionof 87.8% in developing a trained model, reducing roughly 20 hours of manual labeling, to only 2.4 hours.This efficiency gain is associated with only a 6.7% decrease in overall mean average precision (mAP) whentraining multispectral models via MATT, compared to a manually labeled dataset. We consider this anacceptable level of precision loss when considering the time saved during training, especially for rapidlyprototyping experimental modeling methods. This research greatly contributes to the study of multispectralobject detection by providing a novel and open-source method to rapidly segment, label, and trainmultispectral object detection models with minimal human interaction. Future research needs to focus onapplying these methods to (i) space-based multispectral, and (ii) drone-based hyperspectral imagery.