pshinde612/SEW-TWIST

--- language: - en tags: - robotics - teleoperation - humanoid-robot - motion-capture - imitation-learning - mujoco - bvh license: apache-2.0 --- # SEW-TWIST G1 Teleoperation Dataset This dataset contains **offline teleoperation trajectories for the Unitree G1 humanoid robot** generated using the **SEW-MIMIC controller** \[1\] and **LaFAN1 BVH motion capture data** \[2\]. The dataset was generated by replaying BVH motion capture sequences through a MuJoCo simulation of the G1 robot and logging the resulting robot state trajectories in a format compatible with **TWIST-style imitation learning pipelines** \[3\]. Each trajectory is stored as a `.pkl` file containing joint states, root pose, and body positions for each simulation frame. --- # Dataset Structure ``` dataset/ │ ├── *.pkl # Motion trajectories ├── metadata.csv # Optional metadata per sequence └── README.md ``` Each `.pkl` file contains a **single trajectory sequence** recorded from BVH playback. --- # Data Format Each `.pkl` file contains a Python dictionary with the following structure: ```python { "fps": float, "root_pos": np.ndarray, "root_rot": np.ndarray, "dof_pos": np.ndarray, "local_body_pos": np.ndarray, "link_body_list": list[str] } ``` --- # Fields ## fps float Frames per second of the recorded trajectory. --- ## root_pos shape: (T, 3) dtype: float32 World-space position of the robot root (pelvis mocap frame). [x, y, z] Units: **meters** --- ## root_rot shape: (T, 4) dtype: float32 Root orientation quaternion in **(x, y, z, w)** format. This is converted from MuJoCo's internal `(w, x, y, z)` ordering during logging. --- ## dof_pos shape: (T, 21) dtype: float32 Joint configuration vector for the robot. The DOF vector is concatenated as: [left_leg, right_leg, torso, left_arm, right_arm] ### DOF Breakdown | Body Part | DOF | |-----------|----| | Left Leg | 6 | | Right Leg | 6 | | Torso | 3 | | Left Arm | 4 | | Right Arm | 4 | Total: **23 DOF** --- ## local_body_pos shape: (T, N, 3) dtype: float32 Local body positions for each link relative to the root frame. Positions are computed as: local = R_root^T * (x_world - root_pos) Where - R_root is the root rotation matrix - x_world is the world position of the body --- ## link_body_list list[str] length = N Names of the robot bodies corresponding to the `local_body_pos` array. Example entries include pelvis, hip joints, knees, ankles, torso, shoulders, elbows, wrists, and hands. --- # Example Usage ```python import pickle with open("trajectory.pkl", "rb") as f: data = pickle.load(f) print(data.keys()) ``` Output: ``` dict_keys([ 'fps', 'root_pos', 'root_rot', 'dof_pos', 'local_body_pos', 'link_body_list' ]) ``` --- # Intended Use This dataset can be used for: - humanoid motion imitation learning - motion retargeting research - policy learning from motion capture - trajectory prediction - humanoid control benchmarking --- # Citation This dataset is released as part of the **SEW-MIMIC project** ```bibtex @misc{sew_mimic, title={A Closed-Form Geometric Retargeting Solver for Upper Body Humanoid Robot Teleoperation}, author={Kong, Chuizheng and Cho, Yunho and Jung, Wonsuhk and others}, year={2026}, note={Project website: https://sew-mimic.com/} } ``` --- # Acknowledgements - **LaFAN1 Motion Capture Dataset** - **MuJoCo Physics Engine** - **SEW Geometric Teleoperation Framework** # References \[1\] **A Closed-Form Geometric Retargeting Solver for Upper Body Humanoid Robot Teleoperation**, Project website: https://sew-mimic.com/ arXiv preprint arXiv:2602.01632, 2026 \[2\] **LaFAN1 Motion Capture Dataset** Ubisoft La Forge. https://github.com/ubisoft/ubisoft-laforge-animation-dataset \[3\] Y. Ze, Z. Chen, J. P. Araújo, Z. Cao, X. B. Peng, J. Wu, and C. K. Liu, **"TWIST: Teleoperated Whole-Body Imitation System,"** Project website: https://yanjieze.com/TWIST/ # Data Sources and Licensing This dataset is generated from the **LaFAN1 Motion Capture Dataset** provided by Ubisoft La Forge. The original dataset is licensed under: **Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0)** See the original repository for details: https://github.com/ubisoft/ubisoft-laforge-animation-dataset Users of this dataset must comply with the license terms of the original dataset.--- 语言： - 英语 标签： - 机器人学 - 遥操作 - 类人机器人（humanoid robot） - 动作捕捉 - 模仿学习 - MuJoCo - BVH 许可证：apache-2.0 --- # SEW-TWIST G1遥操作数据集 本数据集包含针对Unitree G1类人机器人的离线遥操作轨迹，这些轨迹通过SEW-MIMIC控制器[1]与LaFAN1 BVH动作捕捉数据[2]生成。 本数据集的生成流程为：通过基于G1机器人的MuJoCo物理仿真重放BVH动作捕捉序列，并将生成的机器人状态轨迹以兼容TWIST风格模仿学习流水线[3]的格式进行存储记录。 每条轨迹均存储为.pkl文件，其中包含每一仿真帧的关节状态、根节点位姿与肢体位置信息。 --- # 数据集结构 dataset/ │ ├── *.pkl # 运动轨迹文件 ├── metadata.csv # 可选的单序列元数据文件 └── README.md 每个.pkl文件均对应一条由BVH重放录制得到的轨迹序列。 --- # 数据格式 每个.pkl文件均为包含以下结构的Python字典： python { "fps": float, "root_pos": np.ndarray, "root_rot": np.ndarray, "dof_pos": np.ndarray, "local_body_pos": np.ndarray, "link_body_list": list[str] } --- # 字段说明 ## 帧率（fps） 浮点型，录制轨迹的帧率。 ## 根节点位置（root_pos） 形状为(T, 3)，数据类型为float32。机器人根节点（骨盆动作捕捉坐标系）的世界空间位置，格式为[x, y, z]，单位为米。 ## 根节点旋转（root_rot） 形状为(T, 4)，数据类型为float32。根节点朝向四元数，采用(x, y, z, w)格式。该值在记录时已从MuJoCo内部的(w, x, y, z)顺序转换而来。 ## 关节位置（dof_pos） 形状为(T, 21)，数据类型为float32。机器人的关节配置向量。关节自由度（DOF）向量按如下顺序拼接：[左腿、右腿、躯干、左臂、右臂]。 ### 自由度细分 | 身体部位 | 自由度数量 | |-----------|----| | 左腿 | 6 | | 右腿 | 6 | | 躯干 | 3 | | 左臂 | 4 | | 右臂 | 4 | 总计：23个自由度 ## 局部肢体位置（local_body_pos） 形状为(T, N, 3)，数据类型为float32。各肢体相对于根节点坐标系的局部位置。位置计算公式为： `local = R_root^T * (x_world - root_pos)` 其中： - R_root 为根节点旋转矩阵 - x_world 为肢体的世界空间位置 ## 肢体名称列表（link_body_list） 长度为N的字符串列表，对应local_body_pos数组中各肢体的名称。示例条目包括骨盆、髋关节、膝关节、踝关节、躯干、肩关节、肘关节、腕关节以及手部。 --- # 使用示例 python import pickle with open("trajectory.pkl", "rb") as f: data = pickle.load(f) print(data.keys()) 输出： dict_keys([ 'fps', 'root_pos', 'root_rot', 'dof_pos', 'local_body_pos', 'link_body_list' ]) --- # 预期应用场景 本数据集可应用于以下场景： - 类人机器人动作模仿学习 - 动作重定向研究 - 基于动作捕捉的策略学习 - 轨迹预测 - 类人机器人控制基准测试 --- # 引用 本数据集作为SEW-MIMIC项目的一部分发布。 bibtex @misc{sew_mimic, title={A Closed-Form Geometric Retargeting Solver for Upper Body Humanoid Robot Teleoperation}, author={Kong, Chuizheng and Cho, Yunho and Jung, Wonsuhk and others}, year={2026}, note={Project website: https://sew-mimic.com/} } --- # 致谢 - **LaFAN1动作捕捉数据集** - **MuJoCo物理引擎** - **SEW几何遥操作框架** # 参考文献 [1] **面向上半身类人机器人遥操作的闭式几何重定位求解器**， 项目主页：https://sew-mimic.com/ arXiv预印本 arXiv:2602.01632, 2026 [2] **LaFAN1动作捕捉数据集** Ubisoft La Forge. https://github.com/ubisoft/ubisoft-laforge-animation-dataset [3] Y. Ze、Z. Chen、J. P. Araújo、Z. Cao、X. B. Peng、J. Wu 与 C. K. Liu， **"TWIST：遥操作式全身模仿学习系统"**， 项目主页：https://yanjieze.com/TWIST/ # 数据来源与许可 本数据集基于Ubisoft La Forge发布的LaFAN1动作捕捉数据集生成。 原始数据集的许可协议为： **知识共享署名-非商业性使用-禁止演绎4.0国际许可（CC BY-NC-ND 4.0）** 详情请参阅原始仓库： https://github.com/ubisoft/ubisoft-laforge-animation-dataset 本数据集的使用者必须遵守原始数据集的许可协议条款。