OpenCap

OpenCap combines computer vision, deep learning, and musculoskeletal simulation to quantify human movement dynamics from smartphone videos. See our preprint for more description of OpenCap and our validation experiments:Uhlrich SD*, Falisse A*, Kidzinski L*, Ko M, Chaudhari AS, Hicks JL, Delp SL, 2022. OpenCap: 3D human movement dynamics from smartphone videos. biorxiv. https://doi.org/10.1101/2022.07.07.499061. *contributed equally- To start collecting data with OpenCap, visit https://app.opencap.ai.- To find more information about OpenCap, visit https://opencap.ai.- To find the source code for computing kinematics from videos, visit https://github.com/stanfordnmbl/opencap-core- To find code for post-processing OpenCap data and generating dynamic simulations, visit https://github.com/stanfordnmbl/opencap-processingOpenCap comprises an iOS application, a web application, and cloud computing. To collect data, users open an application on two or more iOS devices and pair them with the OpenCap web application. The web application enables users to record videos simultaneously on the iOS devices and to visualize the resulting 3-dimensional (3D) kinematics. In the cloud, 2D keypoints are extracted from multi-view videos using open-source pose estimation algorithms. The videos are time synchronized using cross-correlations of keypoint velocities, and 3D keypoints are computed by triangulating these synchronized 2D keypoints. These 3D keypoints are converted into a more comprehensive 3D anatomical marker set using a recurrent neural network (LSTM) trained on motion capture data. 3D kinematics are then computed from marker trajectories using inverse kinematics and a musculoskeletal model with biomechanical constraints. Finally, kinetic measures are estimated using muscle-driven dynamic simulations that track 3D kinematics.This repository (see Downloads) contains the experimental data used in the validation study. More details on the participant population can be found in our preprint. More details about the specifics of the included data can also be found in the README included in the downloaded folders.1) Lab Validation Data: Population and activities: 10 individuals performing four activities (squats, sit-to-stand, drop vertical jump, and walking) with varied kinematic patterns. Raw data: Marker-based motion capture, ground reaction forces, electromyography from 10 lower-extremity muscles, RGB video from 5 cameras.Processed data: OpenSim models, inverse kinematics, inverse dynamics, muscle driven simulations.We provide this dataset with and without RGB videos, for file size considerations.2) Field Study Data:Population and activities: 100 individuals performing natural and asymmetric squats.Processed data: OpenSim models, inverse kinematics, muscle driven simulations from OpenCap using two cameras. RGB videos are not provided with this dataset, due to the more restrictive IRB protocol that we used for this portion of the study. <br/><br/>This project includes the following software/data packages: <br/> <ul> <li> <a href="https://simtk.org/frs?group_id=2385#pack_2355">FieldStudy </a> : This package contains data (excluding videos) used for our field study. Please take a look at the README file and project description for details. </li> <li> <a href="https://simtk.org/frs?group_id=2385#pack_2353">LabValidation_withoutVideos </a> : This package contains data (excluding videos) used for our lab validation. Please take a look at the README file and the project description for details. </li> <li> <a href="https://simtk.org/frs?group_id=2385#pack_2354">LabValidation_withVideos </a> : This package contains data (including videos) used for our lab validation. Please take a look at the README file and project description for details. </li> </ul>

OpenCap融合计算机视觉、深度学习和骨骼肌模拟技术，旨在从智能手机视频中量化人体运动动力学。详见我们的预印本，其中对OpenCap及其验证实验进行了详细描述：Uhlrich SD*, Falisse A*, Kidzinski L*, Ko M, Chaudhari AS, Hicks JL, Delp SL, 2022. OpenCap：从智能手机视频中提取三维人体运动动力学。biorxiv. https://doi.org/10.1101/2022.07.07.499061. *均等贡献- 若要使用OpenCap收集数据，请访问https://app.opencap.ai。- 欲了解更多关于OpenCap的信息，请访问https://opencap.ai。- 若要获取从视频中计算运动学的源代码，请访问https://github.com/stanfordnmbl/opencap-core。- 若要获取OpenCap数据的后处理代码和生成动态模拟的代码，请访问https://github.com/stanfordnmbl/opencap-processing。OpenCap包括iOS应用程序、Web应用程序和云计算服务。为了收集数据，用户需要在两个或更多iOS设备上打开应用程序，并与OpenCap Web应用程序配对。Web应用程序允许用户在iOS设备上同步录制视频，并可视化所得的三维（3D）运动学。在云端，通过使用开源姿态估计算法从多视角视频中提取二维（2D）关键点，并利用关键点速度的互相关函数实现视频的时间同步。通过三角测量这些同步的2D关键点，计算得到3D关键点。利用在运动捕捉数据上训练的循环神经网络（LSTM）将这些3D关键点转换为更为全面的3D解剖标记集。然后，通过逆向动力学和具有生物力学约束的骨骼肌模型，从标记轨迹中计算3D运动学。最后，通过肌肉驱动的动态模拟跟踪3D运动学，估算动力学指标。此存储库（见下载）包含用于验证研究的实验数据。关于参与者群体更多的细节可在我们的预印本中找到。关于所包含数据的详细情况，也可在下载文件夹中包含的README文件中找到。1) 实验室验证数据：参与者及活动：10名个体进行四种活动（深蹲、坐姿到站立、垂直跳跃和行走）并具有不同的运动学模式。原始数据：基于标记的运动捕捉、地面反作用力、来自10个下肢肌肉的肌电图、5个摄像头的RGB视频。处理数据：OpenSim模型、逆向运动学、逆向动力学、肌肉驱动模拟。我们提供包含和不包含RGB视频的数据集，以考虑文件大小。2) 场地研究数据：参与者及活动：100名个体进行自然非对称的深蹲。处理数据：OpenSim模型、逆向运动学、使用两个摄像头的OpenCap肌肉驱动模拟。由于本部分研究使用的IRB协议更为严格，因此未提供RGB视频。<br/><br/>本项目包括以下软件/数据包：<br/><ul> <li><a href="https://simtk.org/frs?group_id=2385#pack_2355">FieldStudy </a>：此包包含用于场地研究的数据（不包括视频）。请参阅README文件和项目描述以获取详细信息。</li> <li><a href="https://simtk.org/frs?group_id=2385#pack_2353">LabValidation_withoutVideos </a>：此包包含用于实验室验证的数据（不包括视频）。请参阅README文件和项目描述以获取详细信息。</li> <li><a href="https://simtk.org/frs?group_id=2385#pack_2354">LabValidation_withVideos </a>：此包包含用于实验室验证的数据（包括视频）。请参阅README文件和项目描述以获取详细信息。</li> </ul>

搜集汇总

背景与挑战

背景概述

OpenCap是一个用于从智能手机视频估计3D人体运动动力学的软件包数据集，结合了计算机视觉、深度学习和肌肉骨骼模拟技术。数据集包含验证研究中的实验数据，具体分为实验室验证数据（涉及10名参与者进行深蹲、坐站、垂直跳跃和行走等活动）和现场研究数据（涉及100名参与者进行自然和非对称深蹲），提供了处理后的运动学和动力学测量结果。