PeopleSansPeople Dataset

In recent years, person detection and human pose estimation have made great strides, helped by large-scale labeled datasets. However, these datasets had no guarantees or analysis of human activities, poses, or context diversity. Additionally, privacy, legal, safety, and ethical concerns may limit the ability to collect more human data. An emerging alternative to real-world data that alleviates some of these issues is synthetic data. However, creation of synthetic data generators is incredibly challenging and prevents researchers from exploring their usefulness. Therefore, we release a human-centric synthetic data generator PeopleSansPeople which contains simulation-ready 3D human assets, a parameterized lighting and camera system, and generates 2D and 3D bounding box, instance and semantic segmentation, and COCO pose labels. Using PeopleSansPeople, we performed benchmark synthetic data training using a Detectron2 Keypoint R-CNN variant [1]. We found that pre-training a network using synthetic data and fine-tuning on target real-world data (few-shot transfer to limited subsets of COCO-person train [2]) resulted in a keypoint AP of 60.37±0.48 (COCO test-dev2017) outperforming models trained with the same real data alone (keypoint AP of 55.80) and pre-trained with ImageNet (keypoint AP of 57.50). This freely-available data generator should enable a wide range of research into the emerging field of simulation to real transfer learning in the critical area of human-centric computer vision.

近年来，得益于大规模标注数据集的助力，人体检测与姿态估计技术取得了显著进步。然而，这些数据集并未提供关于人类活动、姿态或情境多样性的保障与分析。此外，隐私、法律、安全及伦理方面的顾虑可能限制了收集更多人类数据的可能性。作为一种缓解上述问题的替代方案，合成数据正逐渐兴起。然而，合成数据生成器的创建极具挑战性，阻碍了研究人员对其效用性的探索。鉴于此，我们发布了一款以人为中心的合成数据生成器PeopleSansPeople，它包含了模拟准备就绪的3D人体资产、参数化的光照和摄像机系统，并生成2D和3D边界框、实例和语义分割，以及COCO姿态标签。借助PeopleSansPeople，我们采用Detectron2 Keypoint R-CNN变体[1]进行了基准合成数据训练。我们发现，利用合成数据进行网络预训练并在目标真实数据上微调（少样本迁移至COCO-person训练的有限子集[2]）的结果，其关键点AP达到了60.37±0.48（COCO测试-dev2017），超越了仅使用相同真实数据进行训练的模型（关键点AP为55.80）以及使用ImageNet预训练的模型（关键点AP为57.50）。这款免费可用的数据生成器应能促进对模拟至真实迁移学习这一关键领域新兴领域的广泛研究。