WujiTacData

# WujiTacData Tactile dexterous manipulation dataset in the [Open-X-Tactile](https://open-x-tactile.github.io/) (OXT) / FTP-1 zarr format. Data format, key names and coordinate conventions: see the OXT website and the [FTP-1 data processing README](https://github.com/michaelyuancb/ftp1-policy/blob/main/data_processing/README.md). ## Summary - Sub-dataset: `WujiTacData` (folder `WujiTacData/`) - Embodiment: two Wuji Hand 2 dexterous hands (20 joints each) on Tianji 7-DoF arms, teleoperated - Tactile sensor: `WujiHand2`, fingertip tactile on all ten fingertips (type `state`) - Cameras: head camera (`camera_ego_rgb`) and two wrist cameras, 224x224 RGB, 30 Hz - 45 tasks, 9,246 trajectories, 10,636,917 frames (98.4 h) - 71 zip files, 2.81 TB ## Layout ``` WujiTacData/ WujiTacData/ wuji_<task>[_<batch>][_part<k>of<n>]+<instruction>+.zarr.zip (one zarr per zip) README.md ``` Each zip holds one zarr directory. Extract all of them into one domain folder, which the FTP-1 loader reads as a single domain: ```bash mkdir -p data/WujiTacData for z in WujiTacData/*.zarr.zip; do unzip -q "$z" -d data/WujiTacData/; done ``` The zips are uncompressed (stored), so a zarr can also be read in place with `zarr.storage.ZipStore`. ## Keys (per zarr, `data/`) | key | shape | content | |---|---|---| | `timestamps` | (T,) | seconds from episode start, 30 Hz | | `camera_ego_rgb` | (T, 224, 224, 3) | head camera RGB | | `{side}_wrist_camera_rgb` | (T, 224, 224, 3) | wrist camera RGB | | `{side}_arm_joints` | (T, 7) | arm joint commands (rad) | | `{side}_hand_joints` | (T, 20) | hand joint commands (rad) | | `{side}_hand_joints_idx` | (T, 20) | UAS slot ids `[1,26,2,3, 7,6,8,9, 12,11,13,14, 17,16,18,19, 22,21,23,24]` | | `{side}_wrist_pose` | (T, 6) | wrist pose (xyz + rotation vector) in the head-camera optical frame (x right, y down, z forward) | | `camera_ego_pose` | (T, 6) | zeros (the world frame is the head-camera frame) | | `{side}_tactile_data_fingertips` | (T, 5, 3) | per-fingertip force fx, fy, fz (N); thumb, index, middle, ring, pinky | | `{side}_tactile_{area,sensor,type}_fingertips` | | areas 0-4, `WujiHand2`, `state` | | `sub_task_instruction` | (T,) | English instruction | `{side}` is `left` or `right`. `meta/episode_ends` marks the episodes. ## Tasks | task | instruction | hands | trajectories | hours | |---|---|---|---:|---:| | `battery_into_bottle` | pick up the bottle with the left hand, drop the battery into it with the right hand, then put the bottle back upright | both hands | 210 | 1.40 | | `battery_strawberry_banana_into_box` | put the battery, strawberry and banana into the white storage box | right hand | 513 | 5.14 | | `bimanual_battery_strawberry_banana_into_box` | alternating hands, put the battery, strawberry and banana into the white storage box | alternating hands | 100 | 1.18 | | `bimanual_battery_strawberry_banana_into_woven_basket` | alternating hands, put the battery, strawberry and banana into the woven basket | alternating hands | 100 | 0.92 | | `bimanual_block_charger_box_into_blue_basket` | alternating hands, put the blue wooden block, charger and mouse box into the blue plastic basket | alternating hands | 100 | 0.99 | | `bimanual_block_charger_box_into_woven_basket` | alternating hands, put the blue wooden block, charger and mouse box into the woven basket | alternating hands | 100 | 0.91 | | `bimanual_can_tea_bottle_into_basket` | alternating hands, put the can, oolong tea bottle and sports bottle into the woven basket | alternating hands | 74 | 0.69 | | `bimanual_can_tea_bottle_into_box` | alternating hands, put the can, oolong tea bottle and sports bottle into the white storage box | alternating hands | 50 | 0.58 | | `bimanual_corn_ball_can_into_box` | alternating hands, put the toy corn, plastic ball and can into the white storage box | alternating hands | 128 | 1.35 | | `bimanual_corn_ball_can_into_woven_basket` | alternating hands, put the toy corn, plastic ball and can into the woven basket | alternating hands | 40 | 0.38 | | `bimanual_cubes_pingpong_into_blue_basket` | alternating hands, put the 3x3 Rubik's cube, 2x2 Rubik's cube and ping-pong ball into the blue plastic basket | alternating hands | 100 | 0.94 | | `bimanual_cubes_pingpong_into_box` | alternating hands, put the 3x3 Rubik's cube, 2x2 Rubik's cube and ping-pong ball into the white storage box | alternating hands | 150 | 1.51 | | `bimanual_cubes_pingpong_into_woven_basket` | alternating hands, put the 3x3 Rubik's cube, 2x2 Rubik's cube and ping-pong ball into the woven basket | alternating hands | 54 | 0.52 | | `bimanual_cup_eraser_spray_into_box` | alternating hands, put the stacking cup, blackboard eraser and spray can into the white storage box | alternating hands | 100 | 1.09 | | `bimanual_cup_eraser_spray_into_woven_basket` | alternating hands, put the stacking cup, blackboard eraser and spray can into the woven basket | alternating hands | 88 | 0.83 | | `bimanual_cups_into_box` | alternating hands, put the light blue, light green and light yellow cups into the white storage box | alternating hands | 100 | 0.94 | | `bimanual_cups_into_woven_basket` | alternating hands, put the light blue, light green and light yellow cups into the woven basket | alternating hands | 66 | 0.62 | | `bimanual_food_into_box` | alternating hands, put the broccoli, chicken nugget and egg into the white storage box | alternating hands | 108 | 1.16 | | `bimanual_food_into_woven_basket` | alternating hands, put the broccoli, chicken nugget and egg into the woven basket | alternating hands | 87 | 0.99 | | `bimanual_fruits_into_box` | alternating hands, put the apple, orange and lemon into the white storage box | alternating hands | 111 | 1.18 | | `bimanual_fruits_into_woven_basket` | alternating hands, put the apple, orange and lemon into the woven basket | alternating hands | 86 | 0.85 | | `bimanual_items_into_blue_basket` | alternating hands, put the gum bottle, hair clip and clothespin into the blue plastic basket | alternating hands | 100 | 0.96 | | `bimanual_items_into_box` | alternating hands, put the gum bottle, hair clip and clothespin into the white storage box | alternating hands | 48 | 0.48 | | `bimanual_items_into_woven_basket` | alternating hands, put the gum bottle, hair clip and clothespin into the woven basket | alternating hands | 95 | 0.91 | | `bimanual_vegetables_into_box` | alternating hands, put the tomato, cucumber and carrot into the white storage box | alternating hands | 143 | 1.57 | | `bimanual_vegetables_into_woven_basket` | alternating hands, put the tomato, cucumber and carrot into the woven basket | alternating hands | 96 | 0.92 | | `block_charger_box_into_basket` | put the blue wooden block, charger and mouse box into the blue plastic basket | right hand | 569 | 5.51 | | `block_charger_box_into_woven_basket` | put the blue wooden block, charger and mouse box into the woven basket | right hand | 190 | 1.71 | | `bottle_into_tape_roll` | place the bottle into the tape roll | right hand | 35 | 0.16 | | `corn_ball_can_into_box` | put the toy corn, plastic ball and can into the white storage box | right hand | 370 | 3.47 | | `cubes_pingpong_into_blue_basket` | put the 3x3 Rubik's cube, 2x2 Rubik's cube and ping-pong ball into the blue plastic basket | right hand | 600 | 5.30 | | `cup_eraser_spray_into_basket` | put the stacking cup, blackboard eraser and spray can into the woven basket | right hand | 479 | 4.55 | | `cup_pyramid` | stack the six cups into a pyramid: three on the bottom, two in the middle, one on top | both hands | 169 | 2.45 | | `cups_into_basket` | put the light blue, light green and light yellow cups into the woven basket | right hand | 566 | 5.38 | | `deal_cards` | hold a face-down deck in the left hand, deal three cards one by one with the right hand, then turn each of them face up | both hands | 210 | 4.15 | | `food_into_basket` | put the broccoli, chicken nugget and egg into the woven basket | right hand | 564 | 5.32 | | `food_into_blue_basket` | put the broccoli, chicken nugget and egg into the blue plastic basket | right hand | 140 | 1.52 | | `fruits_into_box` | put the apple, orange and lemon into the white storage box | right hand | 462 | 4.39 | | `items_into_blue_basket` | put the gum bottle, hair clip and clothespin into the blue plastic basket | right hand | 600 | 5.40 | | `key_into_box` | pick up the key with a lateral pinch between the thumb and index finger and put it into the white storage box | right hand | 157 | 0.58 | | `lego_assembly` | hold the base plate with the left hand and use the right hand to stack the blue 2x4 brick, the yellow 2x2 brick and the black 1x2 brick onto it in order | both hands | 202 | 5.17 | | `plug_charger` | hold the socket with the left hand and plug the charger into it with the right hand | both hands | 152 | 1.49 | | `screw_nut` | hold the base with the left hand and screw the nut onto the bolt with two fingers of the right hand | both hands | 206 | 7.07 | | `tape_roll_stack` | place the small tape roll on the large tape roll | right hand | 67 | 0.25 | | `vegetables_into_box` | put the tomato, cucumber and carrot into the white storage box | right hand | 561 | 5.51 | Pick-and-place tasks carry a per-episode instruction with the pick order (and hand) of that episode. ## Links - Dataset: https://www.modelscope.cn/datasets/Imnondeersty/WujiTacData - Open-X-Tactile: https://open-x-tactile.github.io/