Fast animal pose estimation using deep neural networks

Recent work quantifying postural dynamics has attempted to define the
repertoire of behaviors performed by an animal. However, a major drawback
to these techniques has been their reliance on dimensionality reduction of
images which destroys information about which parts of the body are used
in each behavior. To address this issue, we introduce a deep
learning-based method for pose estimation, LEAP (LEAP Estimates Animal
Pose). LEAP automatically predicts the positions of animal body parts
using a deep convolutional neural network with as little as 10 frames of
labeled data for training. This framework consists of a graphical
interface for interactive labeling of body parts and software for training
the network and fast prediction on new data (1 hr to train, 185 Hz
predictions). We validate LEAP using videos of freely behaving fruit flies
(Drosophila melanogaster) and track 32 distinct points on the body to
fully describe the pose of the head, body, wings, and legs with an error
rate of <3% of the animal's body length. We recapitulate a number
of reported findings on insect gait dynamics and show LEAP's
applicability as the first step in unsupervised behavioral classification.
Finally, we extend the method to more challenging imaging situations
(pairs of flies moving on a mesh-like background) and movies from freely
moving mice (Mus musculus) where we track the full conformation of the
head, body, and limbs.