FineAction

时间动作定位（TAL）是视频理解中一个重要且具有挑战性的问题。然而，大多数现有的 TAL 基准测试都是建立在动作类的粗粒度之上的，这在这项任务中表现出两个主要限制。首先，粗略的动作可以使定位模型在高级上下文信息中过拟合，而忽略视频中的原子动作细节。其次，粗略的动作类别通常会导致时间边界的模糊注释，这不适合时间动作定位。为了解决这些问题，我们开发了一种新颖的大规模细粒度视频数据集，称为 FineAction，用于时间动作定位。 FineAction 总共包含 106 个动作类别的 103K 时间实例，在 17K 未修剪的视频中进行了注释。 FineAction由于其具有丰富多样的精细动作类、多实例的密集标注以及不同类的共现动作的鲜明特点，为时间动作定位带来了新的机遇和挑战。为了对 FineAction 进行基准测试，我们系统地研究了几种流行的时间定位方法的性能，并深入分析了短期和细粒度实例对时间动作定位的影响。我们相信 FineAction 可以推进时间动作定位及其他方面的研究。
我们的 FineAction 是一个适合训练深度学习模型的大规模数据集，包含 103,324 个实例，总共 705 个视频小时。因此，FineAction 中的实例数高达每个视频 6.17 个和每个类别 975 个。

Temporal Action Localization (TAL) is an important and challenging problem in video understanding. However, most existing TAL benchmarks are built on coarse-grained action classes, which imposes two major limitations on this task. First, coarse-grained actions can lead localization models to overfit to high-level contextual information while ignoring atomic action details in videos. Second, coarse-grained action categories usually result in ambiguous annotations of temporal boundaries, which is unsuitable for temporal action localization. To address these issues, we develop a novel large-scale fine-grained video dataset named FineAction for temporal action localization. FineAction contains a total of 103K temporal instances across 106 action classes, annotated on 17K untrimmed videos. Characterized by its rich and diverse fine-grained action classes, dense multi-instance annotations, and prominent co-occurring actions across different categories, FineAction brings new opportunities and challenges for temporal action localization. To benchmark FineAction, we systematically evaluate the performance of several popular temporal localization methods and conduct in-depth analysis of the impacts of short-term and fine-grained instances on temporal action localization. We believe that FineAction can advance research in temporal action localization and beyond.
Our FineAction is a large-scale dataset suitable for training deep learning models, containing 103,324 instances with a total of 705 video hours. Therefore, the number of instances in FineAction reaches up to 6.17 per video and 975 per category.

搜集汇总

背景与挑战

背景概述

FineAction是一个大规模细粒度视频数据集，专注于时间动作定位（TAL），包含106个动作类别、103K时间实例和17K未修剪视频，总大小为482.9GB。该数据集具有细粒度动作类、多实例密集标注和共现动作的特点，旨在解决粗粒度动作定位中的过拟合和边界模糊问题，适用于训练深度学习模型以推进视频理解研究。