ML assignment example dataset

A cleaned version of the example dataset is available (in zip file format). Please download it here, The example dataset was generated in a simulated environment, in which we know the ground truths motion of all the key points. Three cameras were set up to record the motion: one external camera (generates cam_ext.xxxxxxx.jpeg data), and two feet (left/right) attached cameras (generate cam_r.xxxxxxx.jpeg and cam_l.xxxxxxx.jpeg data). All of them are in the images folder. Images were generated at 100 fps. You can stack the images together to generate the videos. The key points annotations are in two formats: 3d_keypoints (absolute 3d location of key points in the global coordinate), projected_2D_keypoints (the location of key points in the projected camera planes, specifically xxxxxxx_cam_ext.csv; xxxxxxx_cam_l.csv; xxxxxxx_cam_r.csv). For instance, the entry: “ankle_li, 140.19, 165.56” represents the 2D coordinates for the left inner ankle key point. All the frames between camera images and the key points annotations are synchronized. For example, 0001572_cam_l.csv is the projections at frame 1527 into the left foot camera. This assignment focuses on continuously tracking the key points through the video, specifically for the feet-attached cameras (cam_l and cam_r), where occlusion happens very often. From the suggested model (CoTracker), you will try to figure out a solution that can handle our example data case. If you have extra efforts (within the 20 hours), you can also try to figure out who to generate the 3D location of the key points from the cam_l and cam_r key points estimations. The 3D key points locations are the important information for our pipeline to estimate the 3D body posture. The video folder contains example videos of the three camera view angles.本示例数据集的清理版本已以zip压缩包格式提供，请点击此处下载。 该示例数据集生成于模拟环境中，所有关键点的真实运动状态均为已知。 本次实验共架设三台摄像机采集运动数据：一台外置摄像机（生成cam_ext.xxxxxxx.jpeg格式数据），以及两台分别搭载于左脚、右脚的摄像机（分别生成cam_r.xxxxxxx.jpeg与cam_l.xxxxxxx.jpeg格式数据）。所有图像文件均存储于images文件夹内。 图像采集帧率为100 fps，可通过堆叠图像序列生成对应视频。 关键点标注包含两种格式：三维关键点（3d_keypoints），即关键点在全局坐标系下的绝对三维位置；以及投影二维关键点（projected_2D_keypoints），即关键点在相机成像平面上的投影坐标，对应标注文件分别为xxxxxxx_cam_ext.csv、xxxxxxx_cam_l.csv与xxxxxxx_cam_r.csv。例如条目“ankle_li, 140.19, 165.56”代表左侧内踝关节关键点的二维坐标。 摄像机图像与关键点标注的所有帧均已完成同步对齐。例如，文件0001572_cam_l.csv对应第1527帧在左脚摄像机下的投影结果。 本任务聚焦于通过视频对关键点进行连续跟踪，尤其针对足部搭载摄像机（cam_l与cam_r）的场景——此类场景中遮挡问题频发。请基于推荐模型CoTracker，设计可适配本示例数据集的解决方案。 若您有额外时间（建议不超过20小时），还可尝试探索如何通过cam_l与cam_r的关键点估计结果生成三维关键点位置。三维关键点位置是我们的处理流水线（pipeline）估算人体三维姿态的核心信息。 video文件夹中包含三个摄像机视角的示例视频。