TAP-Vid

TAP-Vid是视频以及点轨道的数据集，可以手动注释或从模拟器获得。目的是评估对任何固体物理表面上的任何可跟踪点的跟踪。算法在某个帧上接收单个查询点，并且必须产生轨道的其余部分，即，包括该点已移动到 (如果可见) 的位置，以及在每隔一帧上是否可见。这需要点级精度 (不同于先前的盒和段跟踪工作)，可能在可变形表面上 (不同于运动的结构) 长期 (不同于光流) 潜在地对任何对象 (即类不可知的，不同于先前的类特定的关键点跟踪在人类上)。

TAP-Vid is a dataset consisting of videos and point tracks, which can be manually annotated or acquired from simulators. Its core objective is to evaluate the tracking performance of any trackable point on any solid physical surface. Given a single query point in a specified frame, the algorithm is required to generate the entire remaining portion of the track: namely, the position the point has moved to (if it remains visible) and its visibility status on every other frame. This task demands point-level precision, representing a departure from prior box or segment tracking research. It supports tracking on deformable surfaces (unlike structure from motion methods), enables long-duration tracking (unlike optical flow approaches), and is inherently class-agnostic for arbitrary objects, differing from prior class-specific keypoint tracking systems that focus exclusively on human subjects.

搜集汇总

背景与挑战

背景概述

TAP-Vid是一个用于评估视频中可跟踪点轨道的数据集，支持手动注释和模拟器生成，强调点级精度和类不可知性。该数据集由DeepMind和牛津大学于2022年发布，适用于长期跟踪和可变形表面的研究。