ML assignment example dataset

A cleaned version of the example dataset is available (in zip file format). Please download it here, The example dataset was generated in a simulated environment, in which we know the ground truths motion of all the key points. Three cameras were set up to record the motion: one external camera (generates cam_ext.xxxxxxx.jpeg data), and two feet (left/right) attached cameras (generate cam_r.xxxxxxx.jpeg and cam_l.xxxxxxx.jpeg data). All of them are in the images folder. Images were generated at 100 fps. You can stack the images together to generate the videos. The key points annotations are in two formats: 3d_keypoints (absolute 3d location of key points in the global coordinate), projected_2D_keypoints (the location of key points in the projected camera planes, specifically xxxxxxx_cam_ext.csv; xxxxxxx_cam_l.csv; xxxxxxx_cam_r.csv). For instance, the entry: “ankle_li, 140.19, 165.56” represents the 2D coordinates for the left inner ankle key point. All the frames between camera images and the key points annotations are synchronized. For example, 0001572_cam_l.csv is the projections at frame 1527 into the left foot camera. This assignment focuses on continuously tracking the key points through the video, specifically for the feet-attached cameras (cam_l and cam_r), where occlusion happens very often. From the suggested model (CoTracker), you will try to figure out a solution that can handle our example data case. If you have extra efforts (within the 20 hours), you can also try to figure out who to generate the 3D location of the key points from the cam_l and cam_r key points estimations. The 3D key points locations are the important information for our pipeline to estimate the 3D body posture. The video folder contains example videos of the three camera view angles.