Minimal data Fig 14 specks.

PurposeTo develop and validate a deep learning-based model for automated evaluation of mammography phantom images, with the goal of improving inter-radiologist agreement and enhancing the efficiency of quality control within South Korea’s national accreditation system.Materials and methodsA total of 5,917 mammography phantom images were collected from the Korea Institute for Accreditation of Medical Imaging (KIAMI). After preprocessing, 5,813 images (98.2%) met quality standards and were divided into training, test, and evaluation datasets. Each image included 16 artificial lesions (fibers, specks, masses) scored by certified radiologists. Images were preprocessed, standardized, and divided into 16 subimages. An EfficientNetV2_L-based model, selected for its balance of accuracy and computational efficiency, was used to predict both lesion existence and scoring adequacy (score of 0.0, 0.5, 1.0). Model performance was evaluated using accuracy, F1-score, area under the curve (AUC), and explainable AI techniques.ResultsThe model achieved classification accuracy of 87.84%, 93.43%, and 86.63% for fibers (F1: 0.7292, 95% bootstrap CI: 0.711, 0.747), specks (F1: 0. 7702, 95% bootstrap CI: 0.750, 0.791), and masses (F1: 0.7594, 95% bootstrap CI: 0.736, 0.781), respectively. AUCs exceeded 0.97 for 0.0-score detection and above 0.94 for 0.5-score detection. Notably, the model demonstrated strong discriminative capability in 1.0-score detection across all lesion types. Model interpretation experiments confirmed adherence to guideline criteria: fiber scoring reflected the “longest visible segment” rule; speck detection showed score transitions at two and four visible points; and mass evaluation prioritized circularity but showed some size-related bias. Saliency maps confirmed alignment with guideline-defined lesion features while ignoring irrelevant artifacts.ConclusionThe proposed deep learning model accurately assessed mammography phantom images according to guideline criteria and achieved expert-level performance. By automating the evaluation process, the model can improve scoring consistency and significantly enhance the efficiency and scalability of quality control workflows.研究目的：开发并验证一款基于深度学习的模型，用于乳腺摄影模体图像（mammography phantom images）的自动化评估，旨在提升放射科医师间的评分一致性，并优化韩国国家认证体系内的质量控制流程。 材料与方法：从韩国医学影像认证研究院（Korea Institute for Accreditation of Medical Imaging, KIAMI）共收集到5917张乳腺摄影模体图像。经预处理后，其中5813张（占比98.2%）符合质量标准，被划分为训练集、测试集与验证集。每张图像包含16处人工病变（纤维、斑点、肿块），均由认证放射科医师完成评分。对图像进行预处理、标准化操作，并分割为16张子图像。本研究选用兼顾准确率与计算效率的EfficientNetV2_L模型，用于预测病变存在与否以及评分适配性（评分取值为0.0、0.5、1.0）。模型性能通过准确率、F1分数、曲线下面积（Area Under the Curve, AUC）以及可解释人工智能（explainable AI）技术进行评估。 结果：针对纤维、斑点与肿块三类病变，模型的分类准确率分别为87.84%、93.43%与86.63%，对应的F1分数分别为0.7292（95% Bootstrap置信区间：0.711, 0.747）、0.7702（95% Bootstrap置信区间：0.750, 0.791）与0.7594（95% Bootstrap置信区间：0.736, 0.781）。针对评分为0.0的病变检测，模型的AUC均超过0.97；评分为0.5的病变检测，AUC均高于0.94。值得注意的是，模型在所有病变类型的1.0分评分检测中均展现出优异的区分能力。模型可解释性实验证实其符合指南标准：纤维评分匹配"最长可见段"规则；斑点检测在可见点数为2和4处出现评分跃迁；肿块评估优先考量圆度，但存在一定与尺寸相关的偏倚。显著性热力图（saliency maps）证实模型聚焦于指南定义的病变特征，同时忽略无关伪影。 结论：本研究提出的深度学习模型可依据指南标准准确评估乳腺摄影模体图像，达到了专家级的性能水平。通过自动化评估流程，该模型可提升评分一致性，并显著优化质量控制工作流的效率与可扩展性。