DroneZaic Dataset: a robust end-to-end pipeline for mosaicking freely flown aerial video of agricultural fields

Unoccupied aerial vehicles (UAVs) are increasingly used for
high-throughput phenotyping in quantitative genetics and breeding trials.
In principle, freely flown vehicles would permit real-time flexibility in
identifying and monitoring regions and plants of interest. Mosaicking
multiple images provides a high-resolution global image, and
consumer-grade UAVs offer low cost, ease of flying, and excellent RGB
cameras. However, the vehicles’ inaccurate telemetry complicates
estimating the homographies between pairs of frames during mosaicking, and
accumulated errors distort later portions of the mosaic. Crop fields are
particularly challenging because their regular planting pattern and very
similar plants eliminate the distinctive features that guide mosaicking.
To meet these challenges for a wider range of investigators, we propose
DroneZaic, an end-to-end pipeline that dynamically samples video frames,
automates camera and gimbal calibration, estimates homographies,
and generates mini-mosaics. Together, these
techniques significantly reduce errors in the output mosaics. Our
unsupervised deep learning model component is trained on a comprehensive
video dataset comprising different flight trajectories, maize lines,
growth stages, and synthetic illumination data augmentation, which
involves systematically altering lighting conditions and adding noise to
enhance model generalizability. DroneZaic and its refined CorNetv3, is
more accurate, achieving a 13.1% improvement in APE, 14.11 times
faster than ASIFT, and more robust than our earlier CorNet and CorNetv2.
We demonstrate DroneZaic’s effectiveness and generalizability in computing
accurate mosaics of imagery captured by freely-flown UAVs.