PeopleSansPeople (PeopleSansPeople: A Synthetic Data Generator for Human-Centric Computer Vision)

我们发布了一个以人为中心的合成数据生成器 PeopleSansPeople，其中包含可模拟的 3D 人体资产、参数化照明和摄像系统，并生成 2D 和 3D 边界框、实例和语义分割以及 COCO 姿势标签。使用 PeopleSansPeople，我们使用 Detectron2 Keypoint R-CNN 变体 [1] 执行基准合成数据训练。我们发现，使用合成数据对网络进行预训练并在目标真实世界数据上进行微调（少量镜头转移到 COCO-person train [2] 的有限子集）导致关键点 AP 为 60.37±0.48（COCO 测试-dev2017) 优于仅使用相同真实数据训练的模型（关键点 AP 为 55.80）和使用 ImageNet 预训练的模型（关键点 AP 为 57.50）。这种免费提供的数据生成器应该能够对新兴的模拟领域进行广泛的研究，以在以人为中心的计算机视觉的关键领域进行真正的迁移学习。

We present PeopleSansPeople, a human-centric synthetic data generator equipped with simulatable 3D human assets, parameterized lighting and camera systems, and capable of generating 2D and 3D bounding boxes, instance and semantic segmentation annotations, as well as COCO pose labels. Using PeopleSansPeople, we conducted benchmark synthetic data training with a Detectron2 Keypoint R-CNN variant [1]. We found that pre-training a network on the synthetic data and fine-tuning it on target real-world data via few-shot transfer to a limited subset of the COCO-person train [2] dataset yielded a keypoint AP of 60.37±0.48 on the COCO test-dev2017 benchmark, which outperforms models trained solely on the same real-world data (keypoint AP: 55.80) and ImageNet-pre-trained models (keypoint AP: 57.50). This freely available data generator should enable extensive research in the emerging simulation domain for genuine transfer learning in key areas of human-centric computer vision.

搜集汇总

背景与挑战

背景概述

PeopleSansPeople是一个开源的合成数据生成器，专门用于以人为中心的计算机视觉研究，它提供可模拟的3D人体资产、参数化照明和摄像系统，并能生成2D/3D边界框、实例分割和COCO姿势标签。通过合成数据预训练并结合真实数据微调，该工具在关键点检测任务上表现出优于仅使用真实数据或ImageNet预训练模型的性能。