DroneZaic Dataset: a robust end-to-end pipeline for mosaicking freely flown aerial video of agricultural fields

Unoccupied aerial vehicles (UAVs) are increasingly used for high-throughput phenotyping in quantitative genetics and breeding trials. In principle, freely flown vehicles would permit real-time flexibility in identifying and monitoring regions and plants of interest. Mosaicking multiple images provides a high-resolution global image, and consumer-grade UAVs offer low cost, ease of flying, and excellent RGB cameras. However, the vehicles’ inaccurate telemetry complicates estimating the homographies between pairs of frames during mosaicking, and accumulated errors distort later portions of the mosaic. Crop fields are particularly challenging because their regular planting pattern and very similar plants eliminate the distinctive features that guide mosaicking. To meet these challenges for a wider range of investigators, we propose DroneZaic, an end-to-end pipeline that dynamically samples video frames, automates camera and gimbal calibration, estimates homographies, and generates mini-mosaics. Together, these techniques significantly reduce errors in the output mosaics. Our unsupervised deep learning model component is trained on a comprehensive video dataset comprising different flight trajectories, maize lines, growth stages, and synthetic illumination data augmentation, which involves systematically altering lighting conditions and adding noise to enhance model generalizability. DroneZaic and its refined CorNetv3, is more accurate, achieving a 13.1% improvement in APE, 14.11 times faster than ASIFT, and more robust than our earlier CorNet and CorNetv2. We demonstrate DroneZaic’s effectiveness and generalizability in computing accurate mosaics of imagery captured by freely-flown UAVs.