Pedestrians and Cyclists in Road Traffic: Trajectories, 3D Poses and Semantic Maps

The dataset consists of more than 2300 trajectories of pedestrians and 1000 trajectories of cyclists recorded by a research vehicle of the University of Applied Sciences Aschaffenburg (Kooperative Automatisierte Verkehrssysteme) in urban traffic. In addition to the actual trajectory, the data set contains 3D poses, a representation of the body posture in three-dimensional space, and semantic maps describing the surrounding of the respective vulnerable road user (VRU).

The trajectories were sampled using a sliding window approach and split into a training, validation, and test dataset. Each sample contains the trajectory, 3D poses and semantic maps of the past second, as well as the sought future trajectory and semantic maps for the future 2.52 s. In addition, each pattern is assigned to a current type of motion. The motion types were annotated manually. For a more detailed description of the dataset, please refer to the following publication:

Viktor Kress, Fabian Jeske, Stefan Zernetsch, Konrad Doll, Bernhard Sick: Pose and Semantic Map Based Probabilistic Forecast of Vulnerable Road Users' Trajectories. 2021, arXiv: 2106.02598, https://arxiv.org/abs/2106.02598

We provide files for the training/validation dataset and the test dataset for pedestrians and cyclists, respectively. To read the provided data, unzip the files first. Each file contains a zarr directory. Zarr is a format for the storage of chunked, compressed, N-dimensional arrays (https://zarr.readthedocs.io). To read the data:

import zarr

data = zarr.open()

# get zarr array:
data[]

# uncompress and get data as numpy array (copies the data into RAM)
data[][:]

Each zarr directory contains the following keys:

Key:
pre_trajectories_and_poses: input trajectories of 13 body joint positions, format: [sample, timestep, x,y,z coordinates (first 13 coordinates: x, 14- 26: y, 27:39: z)]
pre_smaps: input semantic maps, format: [sample, timestep (-0.96s, -0.48a, 0.00s)], codes: static obstacles: 0, dynamic obstacles: 1, person: 2, sidewalk: 3, road: 4, walkable vegetation: 5, unknown obstacle: 6, unknown free space: 7, unkown: 8
pos_trajectories: ground truth future trajectories of the head, format: [sample, x,y coordinates (first 63 coordinates: x, 64- 126: y for the timesteps +0.04s, +0.08s, ..., +2.52s))]
pos_smaps: future semantic maps, format: [sample, timestep (+0.44s, +0.96s, +1.48s, +2.00s, 2.52s)]
fold: affiliation to training/validation dataset, format: [sample], codes: test set: 0, validation set: 1, training set: 2
augmentation: affiliation to the augmentation loop (0-2), format: [sample]
move, start, stop, wait, tl, tr: current motion type as boolean arrays, format: [sample]

This work was supported by “Zentrum Digitalisierung.Bayern”. In addition, the work is backed by the project DeCoInt2 , supported by the German Research Foundation (DFG) within the priority program SPP 1835: “Kooperativ interagierende Automobile”, grant numbers DO 1186/1-2 and SI 674/11-2.