Estimating Compositions and Nutritional Values of Seed Mixes based on Vision Transformers

The cultivation of seed mixtures for local pastures is a traditional mixed cropping techniques of cereals and legumes for producing at a low production cost, a balanced animal feed in energy and protein in livestock systems. By considerably improving the autonomy and safety of agricultural systems, as well as reducing their impact on the environment, it is a type of crop that responds favorably both to the evolution of the European regulations on the use phyto-sanitary products, and the expectations of consumers who wish to increase their consumption of organic products. However, farmers find it difficult to adopt it because cereals and legumes do not ripen synchronously and the harvested seeds are heterogeneous, making it more difficult to assess their nutritional value. Many efforts therefore remain to be made to acquire and aggregate technical and economical references to evaluate to what extent the cultivation of seed mixtures could positively contribute to secure and reduce costs on herd feeding. The work presented in this paper proposes to evaluate recent deep learning techniques that could be transferred to an online or smartphone application to automatically estimate the nutritive value of harvested seed mixes to help farmers better managing the yield and thus engage them to promote and contribute to better knowledge of this type of cultivation. For this purpose, we have built an original image dataset containing 4,749 images of seed mixes, covering 11 seed varieties, with which we have compared 2 types of deep learning models. Our results highlight the potential of this method, and show that the best performing model is a recent state-of-the-art Vision Transformer pre-trained with self-supervision (BeiT). It allows an estimation of the nutritive value of seed mixtures with a coefficient of determination \(R^2\) Score of 0.91, which demonstrates the interest of this type of approach, for its possible use on a large scale.

本地牧场用种子混播栽培，是一种传统的禾本科与豆科作物混播技术，可在畜牧生产体系中以较低的生产成本产出能量与蛋白均衡的动物饲料。该技术可显著提升农业系统的自主性与安全性，同时降低其环境影响，既契合欧盟植物检疫产品使用法规的修订方向，也能满足希望提升有机食品消费量的消费者的期待。然而，农户难以推广该技术：禾本科与豆科作物成熟期不同步，收获的种子异质性较强，导致其营养价值评估难度加大。因此，当前仍需开展大量工作，收集并整合技术与经济参考数据，以评估种子混播栽培在保障畜牧饲料供应、降低饲喂成本方面的实际效果。
本文提出的研究旨在评估可迁移至在线平台或智能手机应用的新型深度学习技术，以自动估算收获的种子混播样品的营养价值，帮助农户更好地管控产量，进而推动农户推广该技术并加深对这类栽培模式的认知。为此，我们构建了一套原创性图像数据集，包含4749张种子混播样品图像，涵盖11个种子品种，并基于该数据集对比了两类深度学习模型的性能。研究结果证实了该方法的应用潜力，结果表明，性能最优的模型为采用自监督预训练的前沿视觉Transformer（Vision Transformer，BeiT）。该模型可实现种子混播样品营养价值的估算，决定系数R²得分为0.91，这证明了该方法的应用价值，具备大规模推广的潜力。