A robot dataset of successful and failed placement executions

The dataset contains the following data from successful and failed executions of the Toyota HSR robot placing a book on a shelf.

RGB images from the robot's head camera

Depth images from the robot's head camera

Rendered images of the robot's 3D model from the point of view of the robot's head camera

Force-torque readings from a wrist-mounted force-torque sensor

Joint efforts, velocities and positions

extrinsic and intrinsic camera calibration parameters

frame-level anomaly annotations

The anomalies that occur during execution include:

the manipulated book falling down

books on the shelf being disturbed significantly

camera occlusions

robot being disturbed by an external collision

The dataset is split into a train, validation and test set with the following number of trials:

Train: 48 successful trials

Validation: 6 successful trials

Test: 60 anomalous trials and 7 successful trials