Procedural Human Action Videos

视频中人类动作识别的深度学习正在取得重大进展，但由于其依赖于对大型视频集合进行昂贵的手动标记而减慢了速度。在这项工作中，我们研究了用于动作识别的合成训练数据的生成，因为它最近在各种其他计算机视觉任务中显示出有希望的结果。我们提出了一种可解释的人类动作视频参数生成模型，该模型依赖于现代游戏引擎的程序生成和其他计算机图形技术。我们生成了一个多样化的、现实的、物理上合理的人类动作视频数据集，称为“程序人类动作视频”的 PHAV。它总共包含 39,982 个视频，每个动作有 35 个类别的 1,000 多个示例。我们的方法不限于现有的动作捕捉序列，我们在程序上定义了 14 个合成动作。我们引入了一种深度多任务表示学习架构来混合合成视频和真实视频，即使动作类别不同。我们在 UCF101 和 HMDB51 基准上的实验表明，将我们的大量合成视频与小型真实世界数据集相结合可以提高识别性能，显着优于微调最先进的无监督视频生成模型。

Deep learning for human action recognition in videos has achieved significant progress, but its development is slowed by the reliance on costly manual annotation of large-scale video collections. In this work, we investigate the generation of synthetic training data for action recognition, as it has demonstrated promising results across a variety of other computer vision tasks. We propose an interpretable parametric generative model for human action videos, which relies on procedural generation from modern game engines and other computer graphics techniques. We generate a diverse, realistic, and physically plausible human action video dataset called PHAV, which stands for Procedural Human Action Videos. It comprises a total of 39,982 videos, with over 1,000 examples for each of the 35 action categories. Our approach is not limited to existing motion capture sequences, as we procedurally define 14 synthetic actions. We introduce a deep multi-task representation learning architecture to hybridize synthetic and real-world videos, even across different action categories. Experiments on the UCF101 and HMDB51 benchmarks demonstrate that combining our large-scale synthetic videos with small real-world datasets can improve recognition performance, significantly outperforming fine-tuned state-of-the-art unsupervised video generation models.

搜集汇总

背景与挑战

背景概述

Procedural Human Action Videos（PHAV）是一个通过游戏引擎程序生成的人类动作视频数据集，包含39,982个视频，覆盖35个动作类别，每个类别提供超过1,000个示例。该数据集旨在支持动作识别研究，实验表明其合成视频能与真实数据结合，在UCF101和HMDB51基准上提升识别性能。