The Temporal Shape Dataset

To be able to study both temporal modeling abilities and cross-domain-robustness in a light-weight manner, we propose the Temporal Shape dataset. It is a synthetically created dataset for classification of short clips showing either a square dot or a MNIST digit tracing shapes with their trajectories over time. The dataset has five different trajectory classes (i.e., temporal shapes): circle, line, arc, spiral or rectangle. The task is to recognize which class was drawn by the moving entity across the frames of the sequence. The spatial appearance of the moving object is not correlated with the temporal shape class, and can thus not be employed in the recognition. In the first three domains (2Dot, 5Dot, MNIST), the background is black, and in the last domain (MNIST-bg), the background contains white Perlin noise. The Perlin noise can be more or less fine-grained; scale is regulated by a random parameter. The dataset can be thought of as a heavily scaled-down version of an action template dataset, such as 20BN-Something-something-v2 (Goyal et al., ICCV 2017), entirely stripped of appearance cues. See README.txt for more information on the different zip-files that constitute the dataset.