The MABe22 Benchmarks for Representation Learning of Multi-Agent Behavior

Real-world behavior is often shaped by complex interactions between multiple agents. To scalably study multi-agent behavior, advances in unsupervised and self-supervised learning have enabled many different behavioral representations to be learned from trajectory data. However, such representation learning approaches are generally evaluated on specific datasets and tasks, and it is difficult to compare methods quantitatively to measure progress on representations for behavior analysis. We aim to address this by introducing a large-scale, multi-agent trajectory dataset from real-world behavioral neuroscience experiments that covers a range of behavior analysis tasks. Our dataset consists of common model organisms (mice and flies) in a variety of settings (different strains, lengths of interaction, optogenetic and thermogenetic stimulation), with a subset consisting of expert-annotated behavior labels. Improvements on our dataset corresponds to behavioral representations that work across multiple organisms and is able to capture differences for common behavior analysis tasks.

Sample Python notebooks and evaluator to use our dataset is available at: https://www.aicrowd.com/challenges/multi-agent-behavior-challenge-2022