RGB-Based Behavior Cloning Dataset for Surgical Robotics: 99,522 Episodes of Optimal Demonstrations

Dataset Description:

This dataset contains 99,522 episodes of RGB-based state-action-reward expert demonstrations collected from a reaching task within a surgical robotics simulation environment, LapGym (Scheikl et al.). The data was generated using the LapGym ReachEnv, where a robotic grasper is tasked with reaching a specific point in 3D space. Each episode consists of a series of RGB images (64x64 pixels), corresponding actions, rewards, and terminal flags, designed for training behavior cloning and offline RL algorithms.

This dataset was created for the paper "Assessing Behavior Cloning with RGB Inputs in Surgical Robotics Through Dataset Ablation". The expert demonstrations were collected using an optimal agent, where actions were computed based on the known locations of the grasper and the point of interest.

The specific settings for the ReachEnv environment used to collect the dataset are as follows:

Environment: ReachEnv

Observation Type: RGB

Render Mode: HUMAN

Action Type: CONTINUOUS

Distance to Target Threshold: 0.01

Image Shape: (64, 64)

Frame Skip: 1

Time Step: 0.1

Reward Amounts:

Distance to Target: 0.0

Delta Distance to Target: 0.0

Successful Task: 100.0

Time Step Cost: 0.0

Workspace Violation: 0.0

Sphere Radius: 0.020

Key features of the dataset include:

RGB Inputs: Each episode includes 64x64 RGB frames representing the environment's visual state.

Optimal Demonstrations: All actions represent optimal behavior for completing the reach task.

Sparse Rewards: Rewards are only provided upon successful task completion, offering a challenging learning scenario.

Varied Episode Lengths: Episodes vary in length, depending on how quickly the task is completed.

Applications:

This dataset is designed for research in:

Behavior cloning with RGB image inputs.

Data efficiency and sample efficiency in imitation learning.

Offline reinforcement learning with visual inputs.

Structure:

Observations: Images stored as 64x64 RGB pixel arrays.

Actions: Continuous actions corresponding to the robotic grasper’s movements.

Rewards: Sparse rewards indicating task success.

Terminals: Terminal flags for task completion.

How to Use:

This dataset can be used to train and evaluate offline models for robotic control tasks in conjunction with LapGym, particularly in the domain of surgical robotics. It is especially suited for behavior cloning experiments, offline reinforcement learning, and studies on data efficiency.

Citation:

Please cite this dataset in any publications as:Acs and Zhong (2024). RGB-Based Behavior Cloning Dataset for Surgical Robotics: 99,522 Episodes of Optimal Demonstrations.