DLC networks from: Application of a novel deep learning based 3D videography workflow to bat flight data

Studying the detailed biomechanics of flying animals relies on producing accurate three-dimensional coordinates for key anatomical landmarks. Traditionally, this is achieved through manual digitization of animal videos, a labor-intensive task that grows more so with increasing frame rates and numbers of cameras. In this study, we present a workflow that combines deep learning-powered automatic digitization with intelligent filtering and correction of mislabeled points using 3D information. We tested our workflow using a particularly challenging scenario – bat flight. First, we documented bats flying steadily in a wind tunnel. We compared the results from manually digitizing bats with markers applied to anatomical landmarks against using our automatic workflow on the same bats without markers. In our second test case, we compared manual digitization against our automated workflow for bats exhibiting complex maneuvers in a large flight arena. We found that the variation between the 3D coordinates from our workflow and those from manual digitization was less than a millimeter larger than the variation between 3D coordinates resulting from two different human digitizers. The reduced reliance on manual digitization stemming from this work has the potential to significantly increase the scalability of studies into the detailed biomechanics of animal flight.

Methods
Overview
The data shared here is two DeepLabCut projects, trained on bats in flight. "Brown2021_accuracy_test-DLTdv-2023-01-28" and "Austin2020_accuracy_test-DLTdv-2023-04-07". Both trained and ready for use. They also contain the data on which the networks were trained.

See related article and supplement for more details, and for link to the GitHub repo hosting related code.

Filming
Filming for "Brown2021_accuracy_test-DLTdv-2023-01-28" was performed in the Animal Flight and Aeromechanics (AFAM) Wind Tunnel Facility, Prince Engineering Labs, Brown University.

Filming for "Austin2020_accuracy_test-DLTdv-2023-04-07" was performed at the Austin Bat Refuge.

Digitizing
The training/testing data was digitized using DLTdv and then converted into DeepLabCut compatible format using custom Matlab scripts.