RoHuCAD: Robots and Humans Collaborative Anomaly Detection

RoHuCAD: Robots and Humans Collaborative Anomaly Detection

RoHuCAD is a dataset of human-robot collaboration in a robotic workshop (check workshop_layout.png). Two robots (collaborative manipulator - cobot, autonomous mobile robot - AMR) assist three human operators in assembly of electronic devices.

There are two 8-min long recordings in the dataset. They mostly follow the same scenario, with slightly different anomalies. The data is in ROS Noetic rosbag format.

Included data 

RGBD camera data (color + depth)

3 cameras: Intel Realsense D435i

color and depth data at 6 frames per second

Intrinsic calibration data

Extrinsic calibration data (positions and orientations)

Information about positions of robots

AMR: Ez-Wheel SWD® Starter Kit

Cobot: Universal Robots UR10e

Annotations

Annotations of specific anomalies are included (CSV file with columns: event_id, tstart, tend, event_type, person_id, camera_id)

Gestures / poses

BENT

T-POSE (hands horizontally to the sides)

L+R-UP (both hands up)

RH-UP (right hand up)

LH-UP (left hand up)

SQUAT

HI-POSE (waving)

Unsafe behaviour

Human in robot working area

Standing back to (moving) robot

Looking at phone

Human in the way of AMR

Normal activities

Assembling/Working

Loading/unloading AMR

ROS topics

/tf

/tf_static

/joint_states

cam_ws2_box

/cam_ws2_box/color/camera_info

/cam_ws2_box/color/image_raw/compressed

/cam_ws2_box/depth_registered/camera_info

/cam_ws2_box/depth_registered/image_rect_raw

cam_ta2_ws2

/cam_ta2_ws2/color/camera_info

/cam_ta2_ws2/color/image_raw/compressed

/cam_ta2_ws2/depth_registered/camera_info

/cam_ta2_ws2/depth_registered/image_rect_raw

cam_ta1_ws2

/cam_ta1_ws2/color/camera_info

/cam_ta1_ws2/color/image_raw/compressed

/cam_ta1_ws2/aligned_depth_to_color/camera_info

/cam_ta1_ws2/aligned_depth_to_color/image_raw

Acknowledgement

The work leading to these results has received funding from the European Union’s Horizon Europe research and innovation programme within the ULTIMATE project under the Grant Agreement no 101070162.