Variation in forest root image annotation by experts, novices, and AI

Background The manual study of root dynamics using images requires huge investments of time and resources and is prone to previously poorly quantified annotator bias. AI image-processing tools have been successful in overcoming limitations of manual annotation in homogeneous soils, but their efficiency and accuracy is yet to be widely tested on less homogenous, non-agricultural soil profiles, e.g., that of forests, from which data on root dynamics are key to understanding the carbon cycle. Here, we quantify variance in root length measured by human annotators with varying experience levels. We evaluate the application of a convolutional neural network (CNN) model, trained on a software accessible to researchers without a machine learning background, on a heterogeneous minirhizotron image dataset taken in a multispecies, mature, deciduous temperate forest. Results Less experienced annotators consistently identified more root length than experienced annotators. Root length annotation also varied between experienced annotators. The CNN root length results were neither precise nor accurate, taking ~10% of the time but significantly overestimating root length compared to expert manual annotation (p=0.01). The CNN net root length change results were closer to manual (p=0.08) but there remained substantial variation. Conclusions Manual root length annotation is contingent on the individual annotator. The only accessible CNN model cannot yet produce root data of sufficient accuracy and precision for ecological applications when applied to a complex, heterogeneous forest image dataset. A continuing evaluation and development of accessible CNNs for natural ecosystems is required.研究背景 依托图像开展根系动态的人工研究，需耗费大量时间与资源，且易出现此前难以量化的标注者偏倚问题。人工智能图像处理工具已成功克服均质土壤中人工标注的局限，但它们的效率与精度尚未在非均质、非农业土壤剖面（例如森林土壤剖面）中得到广泛验证；而森林根系动态数据是理解碳循环的关键依据。本研究首先量化了不同经验水平的人工标注者所测得的根长差异；随后评估了卷积神经网络（Convolutional Neural Network, CNN）模型的应用效果——该模型依托一款可供无机器学习背景研究者使用的软件训练完成，测试数据集为取自多物种成熟温带落叶林的异质性微根管图像集。 研究结果 经验较少的标注者测得的根长始终高于经验丰富的标注者；且即便是经验丰富的标注者之间，根长标注结果也存在差异。卷积神经网络的根长检测结果既不精准也不准确，耗时仅约为人工标注的10%，但相较于专家人工标注结果，显著高估了根长（p=0.01）。卷积神经网络得出的净根长变化结果与人工标注结果更为接近（p=0.08），但仍存在较大差异。 研究结论 人工根长标注结果受标注者个体差异影响显著。当前仅有的可及性卷积神经网络模型，应用于复杂异质性森林图像数据集时，仍无法生成满足生态学研究需求的高精度、高精准度根系数据。因此，亟需针对自然生态系统持续评估并开发可及性更强的卷积神经网络工具。