DataSheet_1_Applying machine-learning models to differentiate benign and malignant thyroid nodules classified as C-TIRADS 4 based on 2D-ultrasound combined with five contrast-enhanced ultrasound key frames.docx

ObjectivesTo apply machine learning to extract radiomics features from thyroid two-dimensional ultrasound (2D-US) combined with contrast-enhanced ultrasound (CEUS) images to classify and predict benign and malignant thyroid nodules, classified according to the Chinese version of the thyroid imaging reporting and data system (C-TIRADS) as category 4.
Materials and methodsThis retrospective study included 313 pathologically diagnosed thyroid nodules (203 malignant and 110 benign). Two 2D-US images and five CEUS key frames (“2nd second after the arrival time” frame, “time to peak” frame, “2nd second after peak” frame, “first-flash” frame, and “second-flash” frame) were selected to manually label the region of interest using the “Labelme” tool. A total of 7 images of each nodule and their annotates were imported into the Darwin Research Platform for radiomics analysis. The datasets were randomly split into training and test cohorts in a 9:1 ratio. Six classifiers, namely, support vector machine, logistic regression, decision tree, random forest (RF), gradient boosting decision tree and extreme gradient boosting, were used to construct and test the models. Performance was evaluated using a receiver operating characteristic curve analysis. The area under the curve (AUC), sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), accuracy (ACC), and F1-score were calculated. One junior radiologist and one senior radiologist reviewed the 2D-US image and CEUS videos of each nodule and made a diagnosis. We then compared their AUC and ACC with those of our best model.
ResultsThe AUC of the diagnosis of US, CEUS and US combined CEUS by junior radiologist and senior radiologist were 0.755, 0.750, 0.784, 0.800, 0.873, 0.890, respectively. The RF classifier performed better than the other five, with an AUC of 1 for the training cohort and 0.94 (95% confidence interval 0.88–1) for the test cohort. The sensitivity, specificity, accuracy, PPV, NPV, and F1-score of the RF model in the test cohort were 0.82, 0.93, 0.90, 0.85, 0.92, and 0.84, respectively. The RF model with 2D-US combined with CEUS key frames achieved equivalent performance as the senior radiologist (AUC: 0.94 vs. 0.92, P = 0.798; ACC: 0.90 vs. 0.92) and outperformed the junior radiologist (AUC: 0.94 vs. 0.80, P = 0.039, ACC: 0.90 vs. 0.81) in the test cohort.
ConclusionsOur model, based on 2D-US and CEUS key frames radiomics features, had good diagnostic efficacy for thyroid nodules, which are classified as C-TIRADS 4. It shows promising potential in assisting less experienced junior radiologists.

研究目标：本数据集旨在通过机器学习提取甲状腺二维超声（2D-US）联合超声造影（CEUS）图像的放射组学特征，以分类并预测甲状腺结节的良恶性，所有结节均按照中国版甲状腺影像报告和数据系统（C-TIRADS）划分为4类。
材料与方法：本回顾性研究共纳入313枚经病理确诊的甲状腺结节（其中恶性203枚，良性110枚）。针对每枚结节，选取2张二维超声图像及5帧超声造影关键帧（即造影剂到达后2秒帧、达峰时间帧、达峰后2秒帧、首次灌注帧与二次灌注帧），并通过Labelme工具手动标注感兴趣区域。将每枚结节对应的共7张图像及其标注信息导入达尔文研究平台（Darwin Research Platform）开展放射组学分析。将数据集以9:1的比例随机划分为训练集与测试集。采用6种分类器构建并测试模型，分别为支持向量机、逻辑回归、决策树、随机森林（RF）、梯度提升决策树与极端梯度提升。以受试者工作特征曲线分析评估模型性能，计算曲线下面积（AUC）、灵敏度、特异度、阳性预测值（PPV）、阴性预测值（NPV）、准确率（ACC）及F1分数。由一名低年资放射科医师与一名高年资放射科医师分别阅片，对每枚结节的二维超声图像及超声造影视频进行诊断。随后将二者的AUC与ACC值与本研究最优模型进行对比。
结果：低年资与高年资放射科医师单独采用二维超声、单独采用超声造影、联合采用二维超声与超声造影进行诊断的曲线下面积分别为0.755、0.750、0.784与0.800、0.873、0.890。随机森林（RF）分类器的表现优于其余5种分类器，其训练集AUC为1，测试集AUC为0.94（95%置信区间：0.88~1）。该随机森林模型在测试集上的灵敏度、特异度、准确率、阳性预测值、阴性预测值及F1分数分别为0.82、0.93、0.90、0.85、0.92与0.84。针对C-TIRADS 4类甲状腺结节，融合二维超声与超声造影关键帧放射组学特征的随机森林模型诊断性能与高年资放射科医师相当（AUC：0.94 vs 0.92，P=0.798；ACC：0.90 vs 0.92），且优于低年资放射科医师（AUC：0.94 vs 0.80，P=0.039；ACC：0.90 vs 0.81）。
结论：本研究基于二维超声与超声造影关键帧放射组学特征构建的模型，对C-TIRADS 4类甲状腺结节具有良好的诊断效能，在辅助经验不足的低年资放射科医师方面展现出良好的应用潜力。