HA4M-THMS dataset

Contact: [e-mail]; [e-mail]; [e-mail]; [e-mail] Size: 8.64 MB 1. Description This dataset was derived from the publicly available HA4M dataset (see section 5). The HA4M dataset contains multi-modal data of different human subjects assembling an Epicyclic Gear Train (EGT). During the assembly of the EGT, the human subject has to reach for the different components, which are distributed over the worktable in a fixed configuration. From the HA4M dataset, twelve reaching motions, corresponding to eight different part locations, were extracted for each task demonstration to construct this dataset. This dataset consists of two parts: a container picking dataset and a table picking dataset. For the container picking dataset, containers were used to position the parts of the EGT at the start of the assembly, limiting the variations in the part locations between different demonstrations. For the table picking dataset, no containers were used to position the parts, causing the part locations to vary significantly. For both datasets, only the right-handed reaching motions were considered. To represent the reaching motions, only the position information of the human’s reaching arm was preserved, with current use focusing on the position of the human’s hand, as provided by the Azure Kinect camera and markerless skeleton tracking employed in the HA4M dataset. In this dataset, raw measurement data were used without additional processing (e.g., additional smoothing or filtering). The use of raw measurement data from markerless skeleton tracking introduces additional challenges related to accuracy, noise, and sample rate. Furthermore, the data of some reaching motions contained irregularities (e.g., erroneous measurements) and can be considered outliers, but were intentionally retained to reflect similar conditions encountered during real-world operation. Since the part locations are not explicitly included in the HA4M dataset, the end-points of the hand reaching motions were used to approximate the target locations. For the container picking dataset, the target locations were estimated as the mean of all demonstrated reaching motion end-points corresponding to the same target, which approximates the constant container locations over all task demonstrations. For the table picking dataset, the target locations were determined as the end-points of the hand reaching motions in the current task demonstration, since significant variations in the part locations occur between task demonstrations. Note: For full context, please consult the original HA4M dataset (see section 5). 2. Original acquisition setup See Acquisition_setup.png (source: https://baltig.cnr.it/ISP/ha4m) The sketch shows the original acquisition setup of the HA4M dataset. A Microsoft Azure Kinect camera was placed in front of the operator and the table over which the components were distributed. The camera was placed on a tripod at a height h of 1.54m and a distance d of 1.78m. The camera was tilted down at an angle α of 17 degrees and recorded at 30 frames per second. Note: The data in this dataset were compensated for the angle α of the camera. 3. Content HA4M-THMS │ ├──── Data │ │ │ ├──── Container │ │ - Folder holding the container picking dataset │ │ - Contains 746 csv-files, one for each reaching motion │ │ - Each csv-file is named traj_#####.csv, with ##### representing the number │ │ of the reaching motion │ │ - Each csv-file has 12 columns, corresponding to the shoulder (x, y, z position), │ │ elbow (x, y, z position), wrist (x, y, z position), and hand (x, y, z position) │ │ of the reaching arm │ │ - In each csv-file, the rows provide the consecutive samples │ │ │ ├──── Table │ │ - Folder holding the table picking dataset │ │ - Contains 728 csv-files, one for each reaching motion │ │ - Each csv-file is named traj_#####.csv, with ##### representing the number │ │ of the reaching motion │ │ - Each csv-file has 12 columns, corresponding to the shoulder (x, y, z position), │ │ elbow (x, y, z position), wrist (x, y, z position), and hand (x, y, z position) │ │ of the reaching arm │ │ - In each csv-file, the rows provide the consecutive samples │ │ │ ├──── Targets │ │ │ │ │ ├──── Target_locations_container.csv │ │ │ - File (.csv) containing the mean target locations for the container picking dataset │ │ │ - The csv-file has 3 columns, corresponding to the estimated x, y, and z position │ │ │ of the mean target │ │ │ - The csv-file has 8 rows, corresponding to the 8 considered targets │ │ │ │ │ └──── Target_locations_individual.csv │ │ - File (.csv) containing the individual target locations for both the container │ │ picking dataset and the table picking dataset │ │ - The csv-file has 3 columns, corresponding to the estimated x, y, and z position │ │ of each target │ │ - In the csv-file, the rows correspond to the consecutive reaching motions │ │ │ └──── extra_info.csv │ - File (.csv) providing additional information │ - The csv-file has 4 columns, corresponding to the number, the task ID, the action, │ and the repetition of each reaching motion │ - In the csv-file, each row corresponds to a reaching motion │ └──── Visualize.py - Python file containing example code - The example code demonstrates how the data can be read, interpreted and visualized 4. Prerequisites for example code The provided example code requires an installation of Python3 (developed with Python 3.12.10), including following packages: - numpy (developed with version 2.4.1) - pandas (developed with version 2.3.3) - matplotlib (developed with version 3.10.8) 5. Attribution 5.1. HA4M dataset G. Cicirelli, R. Marani, L. Romeo, M. García Domínguez, J. Heras, A. G. Perri, and T. D’Orazio, “The HA4M dataset: Multi-modal monitoring of an assembly task for human action recognition in manufacturing,” Sci. Data, vol. 9, 2022, Art. no. 745. https://baltig.cnr.it/ISP/ha4m 5.2. Original Creative Commons license https://baltig.cnr.it/ISP/ha4m/-/blob/master/LICENSE?ref_type=heads