DeepWeeds: A Multiclass Weed Species Image Dataset for Deep Learning

DeepWeeds: A Multiclass Weed Species Image Dataset for Deep Learning This repository makes available the source code and public dataset for the work, "DeepWeeds: A Multiclass Weed Species Image Dataset for Deep Learning", published with open access by Scientific Reports: https://www.nature.com/articles/s41598-018-38343-3. The DeepWeeds dataset consists of 17,509 images capturing eight different weed species native to Australia in situ with neighbouring flora. In our work, the dataset was classified to an average accuracy of 95.7% with the ResNet50 deep convolutional neural network. The source code, images and annotations are licensed under CC BY 4.0 license. The contents of this repository are released under an Apache 2 license. Download the dataset images and our trained models images.zip (468 MB) models.zip (477 MB) Due to the size of the images and models they are hosted outside of the Github repository. The images and models must be downloaded into directories named "images" and "models", respectively, at the root of the repository. If you execute the python script (deepweeds.py), as instructed below, this step will be performed for you automatically. <strong>TensorFlow Datasets</strong> Alternatively, you can access the DeepWeeds dataset with TensorFlow Datasets, TensorFlow's official collection of ready-to-use datasets. DeepWeeds was officially added to the TensorFlow Datasets catalog in August 2019. <strong>Weeds and locations</strong> The selected weed species are local to pastoral grasslands across the state of Queensland. They include: "Chinee apple", "Snake weed", "Lantana", "Prickly acacia", "Siam weed", "Parthenium", "Rubber vine" and "Parkinsonia". The images were collected from weed infestations at the following sites across Queensland: "Black River", "Charters Towers", "Cluden", "Douglas", "Hervey Range", "Kelso", "McKinlay" and "Paluma". The table and figure below break down the dataset by weed, location and geographical distribution. <strong>Data organization</strong> Images are assigned unique filenames that include the date/time the image was photographed and an ID number for the instrument which produced the image. The format is like so: <code>YYYYMMDD-HHMMSS-ID</code>, where the ID is simply an integer from 0 to 3. The unique filenames are strings of 17 characters, such as 20170320-093423-1. <strong>labels</strong> The labels.csv file assigns species labels to each image. It is a comma separated text file in the format: <pre><code>Filename,Label,Species ... 20170207-154924-0,jpg,7,Snake weed 20170610-123859-1.jpg,1,Lantana 20180119-105722-1.jpg,8,Negative ... </code></pre> <em>Note: The specific label subsets of training (60%), validation (20%) and testing (20%) for the five-fold cross validation used in the paper are also provided here as CSV files in the same format as "labels.csv".</em> <strong>models</strong> We provide the most successful ResNet50 and InceptionV3 models saved in Keras' hdf5 model format. The ResNet50 model, which provided the best results, has also been converted to UFF format in order to construct a TensorRT inference engine. <pre><code>resnet.hdf5 inception.hdf5 resnet.uff </code></pre> <strong>deepweeds.py</strong> This python script trains and evaluates Keras' base implementation of ResNet50 and InceptionV3 on the DeepWeeds dataset, pre-trained with ImageNet weights. The performance of the networks are cross validated for 5 folds. The final classification accuracy is taken to be the average across the five folds. Similarly, the final confusion matrix from the associated paper aggregates across the five independent folds. The script also provides the ability to measure the inference speeds within the TensorFlow environment. The script can be executed to carry out these computations using the following commands. To train and evaluate the ResNet50 model with five-fold cross validation, use <code>python3 deepweeds.py cross_validate --model resnet</code>. To train and evaluate the InceptionV3 model with five-fold cross validation, use <code>python3 deepweeds.py cross_validate --model inception</code>. To measure inference times for the ResNet50 model, use <code>python3 deepweeds.py inference --model models/resnet.hdf5</code>. To measure inference times for the InceptionV3 model, use <code>python3 deepweeds.py inference --model models/inception.hdf5</code>. <strong>Dependencies</strong> The required Python packages to execute deepweeds.py are listed in requirements.txt. <strong>tensorrt</strong> This folder includes C++ source code for creating and executing a ResNet50 TensorRT inference engine on an NVIDIA Jetson TX2 platform. To build and run on your Jetson TX2, execute the following commands: <pre><code>cd tensorrt/src make -j4 cd ../bin ./resnet_inference </code></pre> <strong>Citations</strong> If you use the DeepWeeds dataset in your work, please cite it as: IEEE style citation: “A. Olsen, D. A. Konovalov, B. Philippa, P. Ridd, J. C. Wood, J. Johns, W. Banks, B. Girgenti, O. Kenny, J. Whinney, B. Calvert, M. Rahimi Azghadi, and R. D. White, “DeepWeeds: A Multiclass Weed Species Image Dataset for Deep Learning,” <em>Scientific Reports</em>, vol. 9, no. 2058, <strong>2</strong> 2019. [Online]. Available: https://doi.org/10.1038/s41598-018-38343-3 ” <strong>BibTeX</strong> <pre><code>@article{DeepWeeds2019, author = {Alex Olsen and Dmitry A. Konovalov and Bronson Philippa and Peter Ridd and Jake C. Wood and Jamie Johns and Wesley Banks and Benjamin Girgenti and Owen Kenny and James Whinney and Brendan Calvert and Mostafa {Rahimi Azghadi} and Ronald D. White}, title = {{DeepWeeds: A Multiclass Weed Species Image Dataset for Deep Learning}}, journal = {Scientific Reports}, year = 2019, number = 2058, month = 2, volume = 9, issue = 1, day = 14, url = "https://doi.org/10.1038/s41598-018-38343-3", doi = "10.1038/s41598-018-38343-3" } </code></pre>**DeepWeeds：面向深度学习的多类杂草物种图像数据集** 本仓库公开了对应研究《DeepWeeds：面向深度学习的多类杂草物种图像数据集》的源代码与公开数据集，该研究由《科学报告（Scientific Reports）》以开放获取形式发表，原文链接：https://www.nature.com/articles/s41598-018-38343-3。 DeepWeeds数据集共包含17509张图像，采集了澳大利亚本土8种不同杂草物种及其伴生植被的原位场景。本研究中，使用ResNet50深度卷积神经网络（ResNet50）对该数据集进行分类，平均分类准确率达到95.7%。 本研究的源代码、图像与标注均采用CC BY 4.0许可协议进行授权，而本仓库的其余内容则遵循Apache 2.0许可协议发布。请下载数据集图像与预训练模型：`images.zip`（468 MB）、`models.zip`（477 MB）。由于图像与模型文件体积较大，它们并未直接托管于GitHub仓库中。请将下载的图像文件解压至仓库根目录下名为`images`的文件夹，预训练模型文件解压至名为`models`的文件夹。若按照下文说明运行Python脚本`deepweeds.py`，该步骤将自动完成。 **TensorFlow数据集（TensorFlow Datasets）** 此外，你可通过TensorFlow Datasets（TensorFlow官方的现成数据集集合）访问DeepWeeds数据集。该数据集已于2019年8月正式加入TensorFlow Datasets数据集目录。 **杂草与采集区域** 本次选用的杂草物种均分布于澳大利亚昆士兰州的牧草地中，具体包括：“中华苹果（Chinee apple）”、“蛇草（Snake weed）”、“马缨丹（Lantana）”、“刺金合欢（Prickly acacia）”、“暹罗草（Siam weed）”、“银胶菊（Parthenium）”、“橡胶藤（Rubber vine）”以及“扁轴木（Parkinsonia）”。所有图像均采集自昆士兰州以下杂草滋生区域：“布莱克河（Black River）”、“查特斯堡（Charters Towers）”、“克拉登（Cluden）”、“道格拉斯（Douglas）”、“赫维岭（Hervey Range）”、“凯尔索（Kelso）”、“麦金莱（McKinlay）”以及“帕鲁马（Paluma）”。下文的表格与图表将按杂草种类、采集区域以及地理分布对数据集进行细分说明。 **数据组织方式** 每张图像均配有唯一文件名，文件名包含图像拍摄的日期/时间以及拍摄设备的ID编号，格式为：`YYYYMMDD-HHMMSS-ID`，其中ID为0到3之间的整数。这类唯一文件名均为17位字符，例如`20170320-093423-1`。 **标签文件** `labels.csv`文件为每张图像分配了物种标签，该文件为逗号分隔的文本文件，格式如下： Filename,Label,Species ... 20170207-154924-0,7,Snake weed 20170610-123859-1.jpg,1,Lantana 20180119-105722-1.jpg,8,Negative ... *注：本文中使用的五折交叉验证所对应的训练集（60%）、验证集（20%）与测试集（20%）的标签子集，也以与`labels.csv`相同格式的CSV文件提供于本仓库中。* **预训练模型** 本仓库提供了表现最优的ResNet50与InceptionV3模型，均以Keras的HDF5模型格式保存。其中取得最佳分类效果的ResNet50模型还被转换为UFF格式，用于构建TensorRT推理引擎（TensorRT）。相关模型文件包括：`resnet.hdf5`、`inception.hdf5`以及`resnet.uff`。 **deepweeds.py脚本** 该Python脚本基于ImageNet预训练权重，在DeepWeeds数据集上训练并评估Keras原生实现的ResNet50与InceptionV3模型。模型性能通过五折交叉验证进行评估，最终分类准确率取五折结果的平均值。同理，相关论文中的最终混淆矩阵也是对五组独立折数的结果进行聚合得到的。该脚本还支持在TensorFlow环境中测量模型的推理速度。你可通过以下命令执行相应计算任务： - 使用五折交叉验证训练并评估ResNet50模型：`python3 deepweeds.py cross_validate --model resnet` - 使用五折交叉验证训练并评估InceptionV3模型：`python3 deepweeds.py cross_validate --model inception` - 测量ResNet50模型的推理速度：`python3 deepweeds.py inference --model models/resnet.hdf5` - 测量InceptionV3模型的推理速度：`python3 deepweeds.py inference --model models/inception.hdf5` **依赖项** 运行`deepweeds.py`所需的Python第三方包已在`requirements.txt`文件中列出。 **tensorrt文件夹** 该文件夹包含用于在NVIDIA Jetson TX2平台上创建并运行ResNet50 TensorRT推理引擎的C++源代码。若要在你的Jetson TX2设备上编译并运行该代码，请执行以下命令： cd tensorrt/src make -j4 cd ../bin ./resnet_inference **引用说明** 若你的研究中使用了DeepWeeds数据集，请按照以下格式进行引用： IEEE格式引用： A. Olsen, D. A. Konovalov, B. Philippa, P. Ridd, J. C. Wood, J. Johns, W. Banks, B. Girgenti, O. Kenny, J. Whinney, B. Calvert, M. Rahimi Azghadi, and R. D. White, "DeepWeeds: A Multiclass Weed Species Image Dataset for Deep Learning", *Scientific Reports*, vol.9, no.2058, 2019年2月. [在线]. 可访问: https://doi.org/10.1038/s41598-018-38343-3 **BibTeX格式** bibtex @article{DeepWeeds2019, author = {Alex Olsen and Dmitry A. Konovalov and Bronson Philippa and Peter Ridd and Jake C. Wood and Jamie Johns and Wesley Banks and Benjamin Girgenti and Owen Kenny and James Whinney and Brendan Calvert and Mostafa {Rahimi Azghadi} and Ronald D. White}, title = {{DeepWeeds: A Multiclass Weed Species Image Dataset for Deep Learning}}, journal = {Scientific Reports}, year = 2019, number = 2058, month = 2, volume = 9, issue = 1, day = 14, url = "https://doi.org/10.1038/s41598-018-38343-3", doi = "10.1038/s41598-018-38343-3" }