Semi-supervised animal action segmentation: head-fixed fly

This dataset features a head-fixed fly engaged in spontaneous behaviors on an air-supported ball (Schaffer et al. 2023, Nature Communications). A camera captured a side view of the fly at 70 Hz. We tracked eight points across the legs and abdomen using five supervised Lightning Pose networks, followed by Ensemble Kalman Smoothing (Biderman et al. 2024, Nature Methods). We consider five behavior categories: still, walk, front groom, back groom, and abdomen move. A single annotator labeled behaviors in chunks of 50 to several hundred contiguous time points using the DeepEthogram GUI (Bohnslav et al. 2021, eLife). Flies often engage in behaviors for longer than 50 frames, so the selected chunks did not contain any transitions from one behavior to another. We use five labeled videos for training, and five for testing. The dataset contains 1.01M frames, only 16.6k of which are labeled.Many thanks to the authors of the "The spatial and temporal structure of neural activity across the fly brain" (Schaffer et al. 2023, Nature Communications) paper who collected and analyzed the original video dataset: Evan S. Schaffer, Neeli Mishra, Matthew R. Whiteway, Wenze Li, Michelle B. Vancura, Jason Freedman, Kripa B. Patel, Venkatakaushik Voleti, Liam Paninski, Elizabeth M. C. Hillman, L. F. Abbott & Richard Axel.