6D-ViCuT: Six degree-of-freedom visual cuboid tracking dataset for manual packing of cargo in warehouses

Visual tracking of objects is a fundamental technology for Industry 4.0, allowing the integration of digital content and real-world objects. The industrial operation known as manual cargo packing can benefit from the visual tracking of objects. No dataset exists to evaluate the visual tracking algorithms on manual packing scenarios. To close this gap, this article presents 6DViCuT, a dataset of images, and 6D pose ground truth of cuboids in a manual packing operation in intralogistics. The initial release of the dataset comprises 28 sessions acquired in a space that recreates a manual packing zone: indoors, area of (6 x 4 x 2) m3, and warehouse illumination. The data acquisition experiment involves capturing images from fixed and mobile RGBD devices and a motion capture system while an operator performs a manual packing operation. Each session contains between 6 and 18 boxes from an available set of 10 types, with each type varying in height, width, depth, and texture. Each session had a duration in the range of 1 to 5 minutes. Each session exhibits operator speed and box type differences (box texture, size heterogeneity, occlusion).

Methods
Data were acquired indoors in an area isolated from sunlight and with tube light illumination. The workspace rebuilds a dispatching zone from a warehouse. The operator performs a manual packing operation while sensors are scanning the scene. Each session was defined to obtain a different combination of four factors: occlusion, the texture of the cargo, cargo size heterogeneity, and speed of the operator. Each of the first three factors has three levels, and the last factor has two levels.

An operator grasps and transports each box from a consolidation to a packing zone. In the meantime, the scene is scanned by three devices:

1. A Microsoft HoloLens-2 (HL2) is attached to the operator’s head and allows the acquisition of RGBD images and camera 6D pose in space.

2. A Microsoft Kinect v2 was fixed with a line of sight to the consolidation zone to acquire RGB-D images.

3. A Phase Space motion capture (MoCap) system was set up to cover the workspace. Two active markers were attached to each box’s top side. Five additional markers were attached to the HL2 device.