LIBS-160K

Radionuclide bone scan is one of the most important modalities for early diagnosis of malignant bone metastases. Despite many computer-assisted bone scanning diagnostic studies have been proposed to solve the problem of time-consuming and laborious manual diagnosis, the absence of publicly accessible data and benchmark has hindered research on foundational pre-training models in this field. To address this gap, we present LIBS-160K, a large-scale image-text paired bone scan dataset comprising 160,362 bone scan images from 6,586 patients. Each image is accompanied by corresponding diagnostic texts in both Chinese and English with image classification labels. Furthermore, the bone scan language–image pre-training (BoneSLIP) model is proposed, which leverages both vision and language to better comprehend bone scan images and text. BoneSLIP achieves good results on various sub-tasks such as bone metastasis prediction, differentiate anatomical regions and image-text retrieval. Our study proposes the first large-scale public dataset of bone scan image-text pairs and vision-language model, providing a foundation for the application of multimodal foundation models in the field of computer-aided bone scan analysis.

放射性核素骨扫描（Radionuclide bone scan）是恶性骨转移早期诊断的最重要影像学手段之一。尽管已有诸多计算机辅助骨扫描诊断研究被提出，以解决人工诊断耗时耗力的问题，但公开可用数据集与基准测试集的缺失，却阻碍了该领域基础预训练模型的研究进展。为填补这一研究空白，本文提出LIBS-160K——一款大规模图像-文本配对骨扫描数据集，包含来自6586名患者的160362张骨扫描图像。每张图像均附带对应的中英文诊断文本与图像分类标签。此外，本文还提出了骨扫描语言-图像预训练（BoneSLIP）模型，该模型同时利用视觉与语言模态，以更好地理解骨扫描图像与文本信息。BoneSLIP在骨转移预测、解剖区域区分以及图像-文本检索等多项子任务中均取得了优异性能。本研究提出了首个大规模公开的骨扫描图像-文本配对数据集与视觉语言模型，为多模态基础模型在计算机辅助骨扫描分析领域的应用奠定了研究基础。