Synthetic images of skeletal motion for pose and kinematics estimation tasks

This dataset contains automatically annotated synthetic images of human kinematics. There are 50,000 images of real motion and 50,000 images of randomized motion. The data is uploaded in ZIP archives of 10,000 images and corresponding annotations, labels, and metadata. The ZIP archives are named "Output_real_X" or "Output_random_X" for images generated from real human kinematics (analyzed from motion capture data) and random-generated kinematics, respectively. The index X ranges from 1 to 5; the archive with index 1 contains the first 10,000 images, index 2 the second 20,000 images, and so on. Each "image" consists of an RGB image, a silhouette mask image, and several body segment mask images in 1024x1024 resolution. Each image contains a single person in a varied pose, skin and clothing texture, background, and viewer perspective. Annotations are generated from musculoskeletal modeling and simulation. They include positions of joints, bodies, and virtual markers of the musculoskeletal model. Positions are given as the 2D coordinates in the image and the depth of the position in meters, measured from the perspective of the viewer. Each position also has a visibility value between 0 and 1, where 0 represents complete occlusion and 1 represents clear visibility. Additionally, each image is annotated with a 2D bounding box enclosing the person. Labels include generalized coordinates of the musculoskeletal model, i.e., joint angles. Annotations and labels are contained in annotations/annotations.csv, which has the following columns: file_names: name of the RGB image without file suffix (.jpg) jp_hip_r_x, jp_hip_r_y, jp_hip_r_z etc: annotations of joint positions all fields start with "jp_" followed by the name of the joint (i.e., "hip_r" which is the right hip) and ending with "_x", "_y", or "_z" x and y coordinates are given in image pixels z is depth of the position and given in meters bp_femur_r_x, bp_femur_r_y, bp_femur_r_z etc: annotations of body positions fields have similar naming convention as above, but starting with "bp_" vm_fibula_head_r_x, vm_fibula_head_r_y, vm_fibula_head_r_z etc: annotations of virtual marker positions fields have similar naming convention as joint positions and body positions, but starting with "vm_" visibility_jp_hip_r, visibility_bp_femur_r, visibility_vm_fibula_head etc: visibility values of annotations bb_x, bb_y, bb_w, bb_h: 2D bounding boxes given in pixels as x, y, width, height gc_hip_flexion_r, gc_hip_adduction_r, gc_hip_rotation_r etc: labels of generalized coordinates, given in radians Additional information is provided in annotations/metadata.csv, which has the following columns: image_name: same as file_names in annotations.csv sex: sex of the human mesh used to create the synthetic person in the image, where 0=male and 1=female weight: index of the body shape morphology of the human mesh, where different indices indicate different body shapes background: index of the HDRI background of the image frame: frame of kinematics used to set the pose of the human in the image clothing: index of the clothing texture of the human in the image skin: index of the skin texture of the human in the image xrot: rotation of the viewer perspective around the horizontal axis, in degrees yrot: rotation of the viewer perspective around the vertical axis, in degrees fov: field of view of the viewer perspective, in degrees See the GitHub repository (https://github.com/jerela/Godosim) if you want to generate your own annotated images. A data paper describing the data generation workflow can be found here. Contact: Jere Lavikainen, [e-mail]