Synthetic Multimodal Dataset using MuJoCo: UR5 Robot Motion

Using the Mujoco environment, we simulated robot trajectory and transitions from one formation to another. Mujoco is a 3D simulator, while Gym serves as an interface to the UR5 robot. The robot has measurement units that allow the acquisition of the angles, positions, and quaternions of the joints and the position of the end-effector. The robot is located on a table with 4 cameras all from the same radius to the center of the robot just rotated by 90° for each of them. Using the described environment, we collected 1999 samples at a rate of 10 samples per second. ################################################################ ################################################################ camera views: id: 0 name: 'camera_0' xmat: array([ 0.70710678, 0.42537261, -0.56485232, -0.70710678, 0.42537261, -0.56485232, 0. , 0.79882181, 0.60156772]) xpos: array([-2., -2., 3.]) id: 1 name: 'camera_1' xmat: array([-0.70710678, 0.42537261, -0.56485232, -0.70710678, -0.42537261, 0.56485232, 0. , 0.79882181, 0.60156772]) xpos: array([-2., 2., 3.]) id: 2 name: 'camera_2' xmat: array([ 0.70710678, -0.42537261, 0.56485232, 0.70710678, 0.42537261, -0.56485232, -0. , 0.79882181, 0.60156772]) xpos: array([ 2., -2., 3.]) id: 3 name: 'camera_3' xmat: array([-0.70710678, -0.42537261, 0.56485232, 0.70710678, -0.42537261, 0.56485232, 0. , 0.79882181, 0.60156772]) xpos: array([2., 2., 3.]) The camera data is stored as .png files with a size of 256x256 ################################################################ ################################################################ The files angles.pt, angular_velocity.pt, angular_acceleration.pt contains information about the motor data of the joints. The angles, velocity, acceleration is the information about the Motor in each joint in following order: ['base_to_lik', 'base_to_rik', 'elbow_joint', 'shoulder_lift_joint', 'shoulder_pan_joint', 'wrist_1_joint', 'wrist_2_joint', 'wrist_3_joint'] ################################################################ ################################################################ The files pose.pt and quaternion.pt contains information about the body data of the robot. The pose of each element is in following order: ['base', 'base_link', 'box_2_link', 'box_link', 'drop_box', 'ee_link', 'forearm_link', 'left_inner_finger', 'left_inner_knuckle', 'right_inner_finger', 'right_inner_knuckle', 'robotiq_85_base_link', 'shoulder_link', 'upper_arm_link', 'world', 'wrist_1_link', 'wrist_2_link', 'wrist_3_link'] ################################################################ ################################################################ The file action.pt contains information about the used action in the corresponding time-step. The action of each element is in following order: ['forearm_T', 'gripper_motor', 'shoulder_lift_T', 'shoulder_pan_T', 'wrist_1_T', 'wrist_2_T', 'wrist_3_T']本数据集基于MuJoCo（Mujoco）仿真环境构建，模拟了机器人的运动轨迹以及编队切换过程。MuJoCo是一款三维物理仿真器，而Gym则作为UR5机器人的控制接口。该机器人配备了测量单元，可采集关节的角度、位置、四元数（quaternion）以及末端执行器的位姿信息。机器人部署于工作台之上，周围搭载4台相机，所有相机与机器人中心的径向距离一致，每台相机之间沿周向旋转90°。依托上述仿真环境，我们以每秒10个样本的采样速率，共采集得到1999组样本数据。 ################################################################ ################################################################ 相机视图参数如下： id: 0，名称：'camera_0'，xmat数组：[ 0.70710678, 0.42537261, -0.56485232, -0.70710678, 0.42537261, -0.56485232, 0. , 0.79882181, 0.60156772]，xpos数组：[-2., -2., 3.] id: 1，名称：'camera_1'，xmat数组：[-0.70710678, 0.42537261, -0.56485232, -0.70710678, -0.42537261, 0.56485232, 0. , 0.79882181, 0.60156772]，xpos数组：[-2., 2., 3.] id: 2，名称：'camera_2'，xmat数组：[ 0.70710678, -0.42537261, 0.56485232, 0.70710678, 0.42537261, -0.56485232, -0. , 0.79882181, 0.60156772]，xpos数组：[ 2., -2., 3.] id: 3，名称：'camera_3'，xmat数组：[-0.70710678, -0.42537261, 0.56485232, 0.70710678, -0.42537261, 0.56485232, 0. , 0.79882181, 0.60156772]，xpos数组：[2., 2., 3.] 所有相机采集的图像均以分辨率为256×256的.png格式文件存储。 ################################################################ ################################################################ angles.pt、angular_velocity.pt、angular_acceleration.pt文件包含机器人各关节的电机数据，其中分别存储关节角度、关节角速度与关节角加速度信息，各关节的顺序如下： ['base_to_lik', 'base_to_rik', 'elbow_joint', 'shoulder_lift_joint', 'shoulder_pan_joint', 'wrist_1_joint', 'wrist_2_joint', 'wrist_3_joint'] ################################################################ ################################################################ pose.pt与quaternion.pt文件包含机器人各构件的位姿数据，其中pose.pt存储各构件的位姿信息，quaternion.pt存储各构件的四元数信息，各构件的顺序如下： ['base', 'base_link', 'box_2_link', 'box_link', 'drop_box', 'ee_link', 'forearm_link', 'left_inner_finger', 'left_inner_knuckle', 'right_inner_finger', 'right_inner_knuckle', 'robotiq_85_base_link', 'shoulder_link', 'upper_arm_link', 'world', 'wrist_1_link', 'wrist_2_link', 'wrist_3_link'] ################################################################ ################################################################ action.pt文件包含对应时间步所使用的控制动作数据，各动作项的顺序如下： ['forearm_T', 'gripper_motor', 'shoulder_lift_T', 'shoulder_pan_T', 'wrist_1_T', 'wrist_2_T', 'wrist_3_T']