Collaborative Task Assembly Dataset (CT-A Dataset)

Description: The Collaborative Task Assembly Dataset (CT-A Dataset) contains multiple collaborative assemblies of the open-source benchmark CT-Benchmark, performed by 5 different operators. CT-A Dataset is a new version of the dataset recorded for Neves et al. with the objective of reducing occlusions that occured when human hands passed behind the robot gripper. The reason for such occlusions in the previous dataset was the sub-optimal robot placement whenever it was waiting for the human to act. Available Data: In total 75 assemblies were recorded by 5 different operators. The handedness and number of assemblies performed by each operator are presented in the following table: Operator index Handedness Number of assemblies 0 Right 55 1 Right 5 2 Right 5 3 Right 5 4 Left 5 The distribution of assemblies by the 5 operators was designed to train and validate models with 50 assemblies from a single operator (operator 0) and to test the trained models with the remaining 25 assemblies (5 for each operator). The dataset was gathered through an Intel D435i RGB-D camera and both RGB and depth videos were recorded at a framerate of 10 fps with a resolution of 1280 x 720 pixels. Each assembly contains 6 sub-assemblies, as in Neves et al., and the sequence by each sub-assembly was performed varied along the videos. Each frame was labelled according to the respective sub-assembly ("B" for Bridge, "D" for Dovetail, "H" for Hospital, "MT" Museum + Triangle, "S" for Snap and "W" for Wheel) or labelled as robot movement ("N/A"). In the dataset there are a total of 75 folders, one for each assembly with the format assembly_(operator index)_(assembly number). Inside each folder there are the following three files: video.mp4 - Contains the RGB data of the assembly. depth.mp4 - Contains the depth data of the assembly. labels.npy - Contains a numpy vector of labels for each frame.