Animal3D Dataset

Accurately estimating the 3D pose and shape is an essential step towards understanding animal behavior, and can potentially benefit many downstream applications, such as wildlife conservation. However, research in this area is held back by the lack of a comprehensive and diverse dataset with high-quality 3D pose and shape annotations. In this paper, we propose Animal3D, the first comprehensive dataset for mammal animal 3D pose and shape estimation. Animal3D consists of 3379 images collected from 40 mammal species, high-quality annotations of 26 keypoints, and importantly the pose and shape parameters of the SMAL model. All annotations were labeled and checked manually in a multi-stage process to ensure highest quality results. Based on the Animal3D dataset, we benchmark representative shape and pose estimation models at: (1) supervised learning from only the Animal3D data, (2) synthetic to real transfer from synthetically generated images, and (3) fine-tuning human pose and shape estimation models. Our experimental results demonstrate that predicting the 3D shape and pose of animals across species remains a very challenging task, despite significant advances in human pose estimation. Our results further demonstrate that synthetic pre-training is a viable strategy to boost the model performance. Overall, Animal3D opens new directions for facilitating future research in animal 3D pose and shape estimation, and is publicly available.

精确估算三维姿态与形态是深入理解动物行为的关键步骤，并可能对众多下游应用如野生动物保护带来潜在益处。然而，该领域的研究受到缺乏全面且多样化的数据集的限制，该数据集需具备高质量的3D姿态与形态标注。在本研究中，我们提出了Animal3D，这是首个针对哺乳动物三维姿态与形态估算的全面数据集。Animal3D包含来自40种哺乳动物的3379张图像，对26个关键点的标注质量上乘，并且包含了SMAL模型的姿态与形态参数。所有标注均经过多阶段的人工标注与检查，以确保达到最高的质量标准。基于Animal3D数据集，我们对比了具有代表性的形状与姿态估算模型在以下三个方面的表现：（1）仅从Animal3D数据中进行的有监督学习，（2）从合成图像到真实图像的迁移学习，以及（3）微调人类姿态与形态估算模型。我们的实验结果表明，尽管在人类姿态估算方面取得了显著进展，但预测不同物种动物的3D形状与姿态仍然是一项极具挑战性的任务。我们的结果还进一步表明，合成预训练是一种可行的策略，用以提升模型性能。总体而言，Animal3D为促进未来动物三维姿态与形态估算研究开辟了新的方向，且该数据集现已公开可用。