Sensitivity examination of YOLOv4 regarding test image distortion and training dataset attribute for apple flower bud classification

Applications of convolutional neural network (CNN)-based object detectors in agriculture have been a popular research topic in recent years. However, complicated agricultural environments bring many difficulties for ground truth annotation as well as potential uncertainties for image data quality. Using YOLOv4 as a representation of state-of-the-art object detectors, this study quantified YOLOv4’s sensitivity against artificial image distortions including white noise, motion blur, hue shift, saturation change, and intensity change, and examined the importance of various training dataset attributes based on model classification accuracies, including dataset size, label quality, negative sample presence, image sequence, and image distortion levels. The YOLOv4 model trained and validated on the original datasets failed at 31.91% white noise, 22.05-pixel motion blur, 77.38° hue clockwise shift, 64.81° hue counterclockwise shift, 89.98% saturation decrease, 895.35% saturation increase, 79.80% intensity decrease, and 162.71% intensity increase with 30% mean average precisions (mAPs) for four apple flower bud growth stages. The performance of YOLOv4 decreased with both declining training dataset size and training image label quality. Negative samples and training image sequence did not make a substantial difference in model performance. Incorporating distorted images during training improved the classification accuracies of YOLOv4 models on noisy test datasets by 13 to 390%. In the context of apple flower bud growth-stage classification, except for motion blur, YOLOv4 is sufficiently robust for potential image distortions by white noise, hue shift, saturation change, and intensity change in real life. Training image label quality and training instance number are more important factors than training dataset size. Exposing models to test-image-alike training images is crucial for optimal model classification accuracies. The study enhances understanding of implementing object detectors in agricultural research.

近年来，基于卷积神经网络（Convolutional Neural Network，CNN）的目标检测器在农业领域的应用一直是热门研究课题。然而，复杂多变的农业环境不仅为真值标注（ground truth annotation）带来诸多挑战，也给图像数据质量带来潜在的不确定性。本研究以YOLOv4作为当前最先进目标检测器的代表，量化分析了YOLOv4对白噪声、运动模糊、色调偏移、饱和度变化以及亮度变化等人工图像失真的敏感性，并基于模型分类准确率，考察了训练数据集的多项属性的重要性，包括数据集规模、标注质量、负样本的存在情况、图像序列以及图像失真程度。在原始数据集上训练并验证的YOLOv4模型，在以下失真条件下其平均精度均值（mean average precision, mAP）降至30%：白噪声强度达31.91%、运动模糊像素量为22.05像素、色调顺时针偏移77.38°、色调逆时针偏移64.81°、饱和度降低89.98%、饱和度提升895.35%、亮度降低79.80%以及亮度提升162.71%，该模型针对4个苹果花芽生长阶段的分类任务性能失效。YOLOv4的性能随训练数据集规模缩减以及训练图像标注质量下降而降低。负样本的存在以及训练图像序列对模型性能未产生显著影响。在训练过程中加入失真图像，可使YOLOv4模型在含噪测试数据集上的分类准确率提升13%至390%。针对苹果花芽生长阶段分类任务而言，除运动模糊外，YOLOv4对实际场景中可能出现的白噪声、色调偏移、饱和度变化以及亮度变化等图像失真具备足够的鲁棒性。训练图像标注质量与训练实例数量，是比训练数据集规模更为关键的影响因素。使模型接触与测试图像风格一致的训练图像，对获得最优的模型分类准确率至关重要。本研究加深了学界对在农业研究中部署目标检测器的认知。