Synthetic Multimodal Dataset using MuJoCo: UR5 Robot Motion

Using the Mujoco environment, we simulated robot trajectory and transitions from one formation        to another. Mujoco is a 3D simulator, while Gym serves as an interface to the UR5 robot.        The robot has measurement units that allow the acquisition of the angles, positions,        and quaternions of the joints and the position of the end-effector. The robot is located on        a table with 4 cameras all from the same radius to the center of the robot just        rotated by 90° for each of them. Using the described environment, we collected 1999 samples         at a rate of 10 samples per second.                ################################################################        ################################################################                camera views:

        id: 0  name: 'camera_0'  xmat: array([ 0.70710678,  0.42537261, -0.56485232, -0.70710678,  0.42537261,       -0.56485232,  0.        ,  0.79882181,  0.60156772])  xpos: array([-2., -2.,  3.])

        id: 1  name: 'camera_1'  xmat: array([-0.70710678,  0.42537261, -0.56485232, -0.70710678, -0.42537261,        0.56485232,  0.        ,  0.79882181,  0.60156772])  xpos: array([-2.,  2.,  3.])

        id: 2  name: 'camera_2'  xmat: array([ 0.70710678, -0.42537261,  0.56485232,  0.70710678,  0.42537261,       -0.56485232, -0.        ,  0.79882181,  0.60156772])  xpos: array([ 2., -2.,  3.])

        id: 3  name: 'camera_3'  xmat: array([-0.70710678, -0.42537261,  0.56485232,  0.70710678, -0.42537261,        0.56485232,  0.        ,  0.79882181,  0.60156772])  xpos: array([2., 2., 3.])

                            The camera data is stored as .png files with a size of 256x256 

                ################################################################        ################################################################                The files angles.pt, angular_velocity.pt, angular_acceleration.pt contains information about        the motor data of the joints. The angles, velocity, acceleration is the information about the         Motor in each joint in following order:            ['base_to_lik', 'base_to_rik', 'elbow_joint', 'shoulder_lift_joint', 'shoulder_pan_joint', 'wrist_1_joint', 'wrist_2_joint', 'wrist_3_joint']                                ################################################################        ################################################################                The files pose.pt and quaternion.pt contains information about        the body data of the robot. The pose of each element is in following order:            ['base', 'base_link', 'box_2_link', 'box_link', 'drop_box', 'ee_link', 'forearm_link', 'left_inner_finger', 'left_inner_knuckle', 'right_inner_finger', 'right_inner_knuckle', 'robotiq_85_base_link', 'shoulder_link', 'upper_arm_link', 'world', 'wrist_1_link', 'wrist_2_link', 'wrist_3_link']                        ################################################################        ################################################################                The file action.pt contains information about the used action in the corresponding time-step.         The action of each element is in following order:            ['forearm_T', 'gripper_motor', 'shoulder_lift_T', 'shoulder_pan_T', 'wrist_1_T', 'wrist_2_T', 'wrist_3_T']